Why Am I Having Problems Getting My E-Mail Delivered?

One of the most frustrating issues facing any e-mail [MX] server operator is non-delivery of e-mail!

 

As ISPs, none of us wants to hear from a customer who is having issues trying to send e-mail, but, the unfortunate fact is more and more of our customers are encountering non-delivery issues and they can be extremely difficult to troubleshoot.

 

There are many reasons an e-mail message might not be delivered, including, but not limited to:

• bad e-mail addresses;
• forgotten passwords;
• having a user's domain or account listed as a source of spam;
• being blocked by the recipient - especially common with AOL users

Here are some of the other common causes of temporary non-delivery issues:

• design - to help prevent spammers from flooding everyone's mailbox with junk mail and spam;
• network congestion - sometimes the networks which carry the data are extremely busy and alternative routing is not available;
• technical issues - sometimes the DNS servers, the master databases which tell all of the other servers using the Internet where everyone is supposed to be located have become corrupted or unreachable;
• software issues - sometimes the software on the other end is non-responsive;
• DDoS attacks - sometimes the receiving e-mail server is bogged down because it is under a "Distributed Denial of Service" attack.  DDoS attacks are extreme situations where someone has illegally targeted a server or network appliance in an attempt to hack or take that machine out of service and can cause major, and sometimes extended, delivery delays;
• hardware issues - sometimes the hardware on the receiving end is in trouble or is undergoing maintenance

If the message was rejected by the receiving MX server you should receive a reason code for the rejection.

 

These delivery codes are based on the Extended SMTP (ESMTP) standards, where X can be 4 or 5, depending on the error type (Persistent Transient or Permanent).

 

Here's a summary of those codes:

• X.1.0 Other address status
• X.1.1 Bad destination mailbox address
• X.2.0 Bad destination system address
• X.1.3 Bad destination mailbox address syntax
• X.1.4 Destination mailbox address ambiguous
• X.1.5 Destination mailbox address valid
• X.1.6 Mailbox has moved
• X.1.7 Bad sender's mailbox address syntax
• X.1.8 Bad sender's system address
   
• X.2.0 Other or undefined mailbox status
• X.2.1 Mailbox disabled, not accepting messages
• X.2.2 Mailbox full
• X.2.3 Message length exceeds administrative limit.
• X.2.4 Mailing list expansion problem
   
• X.3.0 Other or undefined mail system status
• X.3.1 Mail system full
• X.3.2 System not accepting network messages
• X.3.3 System not capable of selected features
• X.3.4 Message too big for system
   
• X.4.0 Other or undefined network or routing status
• X.4.1 No answer from host
• X.4.2 Bad connection
• X.4.3 Routing server failure
• X.4.4 Unable to route
• X.4.5 Network congestion
• X.4.6 Routing loop detected
• X.4.7 Delivery time expired
   
• X.5.0 Other or undefined protocol status
• X.5.1 Invalid command
• X.5.2 Syntax error
• X.5.3 Too many recipients
• X.5.4 Invalid command arguments
• X.5.5 Wrong protocol version
   
• X.6.0 Other or undefined media error
• X.6.1 Media not supported
• X.6.2 Conversion required and prohibited
• X.6.3 Conversion required but not supported
• X.6.4 Conversion with loss performed
• X.6.5 Conversion failed
   
• X.7.0 Other or undefined security status
• X.7.1 Delivery not authorized, message refused
• X.7.2 Mailing list expansion prohibited
• X.7.3 Security conversion required but not possible
• X.7.4 Security features not supported
• X.7.5 Cryptographic failure
• X.7.6 Cryptographic algorithm not supported
• X.7.7 Message integrity failure

As of January, 2013, there's a new reason your customer's e-mail might not be delivered, and many MX operators are not aware of it.  This document will not only help you figure out why you are having issues with e-mail delivery, but will also assist you in ensuring that future e-mail is properly delivered.

 

TO WIT:
 

YAHOO!, Comcast, Google, Hotmail and many other large ISPs adopted new measures to help prevent spam from being delivered to user's in boxes.

 

When you factor these measures in with the requirements of the US CAN SPAM ACT of 2003, and also factor in the European Can Spam Act [which we must, by default, both recognize and abide by in the US because all BLACKBERRY devices use SMTP servers in Vancouver BC Canada; Canada is under British rule and Great Britain is part of the European Union], mailing list management can begin to get really complicated.
 

Based on my experience with YAHOO!, AOL, GMAIL and COMCAST - now the LARGEST ISP in the United States and processes more e-mail than anyone else in the US - and based on the rules, which were put into place by those two operators in January, 2013, these are requirements which should be put into place by every MX server operator:

When you factor these measures with the US CAN SPAM ACT of 2003, and also factor in the European Can Spam Act [which we must, by default, both recognize and abide by in the US because all BLACKBERRY devices which directly access Blackberry e-mail systems use SMTP servers in Vancouver BC Canada; with Canada being under British rule, and Great Britain being a part of the European Union], mailing list management can begin to get really complicated.

Canada's antispam law kicks in on 1 July, 2014
 

Here are a few of the basic requirements which must be implemented to ensure you are in compliance with the new requirements set forth by YAHOO! and COMCAST.

 

By following these requirements you will also be in compliance with all of the other large ISPs:
 

1. Make certain you are in full compliance of ALL THREE can spam acts with every e-mail which leaves your e-mail servers.
    
2. While you and your customers may believe the destination of their e-mail is a US based e-mail account, Internet data can cross international borders without our knowledge.  Once that data crosses a country's border, it is subject to that country's regulations.
   
   Here are links to the three major CAN SPAM ACTS:
    
   1. US CAN SPAM ACT of 2003;
       
   2. European Can Spam Act;
       
   3. Canada's antispam law
       
3. Make certain PORT 587 is OPEN on ALL MAIL SERVERS.
    
   1. The IETF, under RFC 5068, has now PROHIBITED providers from blocking port 587.
       
   2. RFC 5068 clearly says: "Access Providers MUST NOT block users from accessing the external Internet using the SUBMISSION port 587."
       
   3. The complete RFC is available here: http://tools.ietf.org/html/rfc5068
       
4. Make certain you have received ADVANCE PERMISSION from everyone who is on your mailing list in the form of DOUBLE OPT IN.  Because of requirements contained in the European Can Spam Act, it is no longer acceptable to add an address because you have done business with them.
    
5. In any SIGN-UP forms DO NOT ADVANCE CHECK the OPT-IN BOX.
    
   1. The OPT-IN box MUST be SELECTED or COMPLETED by the INDIVIDUAL who might be interested in signing up for an e-mail list.
       
   2. EU regulations prohibit the pre-completion or pre-selection of any field in a newsletter or mailing list sign-up form.
       
   3. Contrary to US antispam regulations, the EU mandate includes those who purchase items from online stores.  They must ACTIVELY INITIATE the SIGN ME UP process by either CHECKING A BOX or ENTERING THEIR E-MAIL ADDRESS into a box - you cannot pre-populate that data.
       
   4. Even though someone makes a purchase from a website, they must still complete the double-opt-in process described below.
       
      1. All sign-ups must include a follow-up e-mail which forces the customer or person signing up for the list to actively complete the DOUBLE-OPT-IN step.
         
         IE: The customer must MAKE THE CHOICE a SECOND TIME to validate their initial sign-up process for the list.
          
      2. To see an example of how this works, go to http://www.chicagonettech.com/subscribenewsletter.asp, and enter your e-mail address.
          
      3. In a few minutes, you will receive a request to CONFIRM your subscription request.
          
      4. Once you click on the link, you will be added.  If you do not complete the confirmation step, you will not be added to the list.
          
      5. NOTE: This is entirely handled by our SmarterMail e-mail server's list serve package which is standard in the Enterprise edition of the software.
          
      6. An alternative to using the SmarterMail list server is to use CONSTANT CONTACT, MAIL CHIMP or one of the other third-party services which provide bulk mailing services and automatically include double-opt-in as part of their sign-up process.
          
      7. Should you choose to use a 3^rd party bulk mailing service, you MUST also include the appropriate SPF INCLUDE statement in your SPF record so bulk messages sent on behalf of any of your hosted domains which choose to use the external bulk mailing services are not rejected because the proper SPF information is not included.
          
6. Make certain you have a VISIBLE, PUBLIC, WORKING PRIVACY PAGE posted on each website which has a domain hosted on your MX server.
   
   YAHOO will actively VERIFY THIS via a human being who will actually open and read your privacy page before they will approve you for the FEEDBACK LOOP [see Item #11 below for more information on Feedback Loops].
   
   See our privacy policy at: http://www.chicagonettech.com/privacy.asp as a sample page and feel free to copy it if you do not already have one.
    
7. All bulk list, sales, and promotional messages must include complete contact information, including: a mailing address and unsubscribe link.
    
8. Make certain ALL e-mail is sent via an AUTHENTICATED E-MAIL ACCOUNT.  Do NOT send e-mail directly to a recipient from a webpage, form or shopping cart.  Instead, setup the form, website or shopping cart to use a designated e-mail address to AUTHENTICATE with your MX server and send the message THROUGH the MX server.
   
   This may mean going back and revisiting the code in some web pages, but it must be done because messages which are not authenticated will be treated like spam and rejected by many MX servers.
   
   OBSERVATIONS:
    
   1. In spite of the inclusion of that information, most AOL users will still report you as SPAM and fail to unsubscribe from your lists.  That's where the FEEDBACK loops come in.
       
   2. The feedback loops will mostly protect you from the brainless AOL users because you will be notified and, if you have properly included an unsubscribe link in the message, will be able to open the attachment and click on the unsubscribe link to remove the brainless user from future mailings.
       
9. Make certain you IMMEDIATELY REMOVE someone from the list if they request such an action - an automated removal format is preferred.
   
   REMEMBER: all AUTO-GENERATED and COMMERCIAL e-mail messages must contain a WORKING opt-out link.
   
   NOTE: You are NOT ALLOWED to require any kind of input or reason in the removal form.
   
   NOTE: You should attempt to auto-populate the e-mail address requesting removal in the removal form, if possible, and CANNOT enforce a removal reason to be entered as part of the removal request.  You can ask that they provide a removal reason, but cannot deny removal because someone refuses to enter a reason.
    
10. You [your hosted customers] MUST AUTO-REMOVE ALL bouncing e-mail addresses.  We auto-remove all users after a list or newsletter bounces 3 or more times.  Again, our MX server package, SmarterMail, takes care of this automatically.
     
11. Make certain you have setup FEEDBACK LOOPS with all of the major providers.  The setup of these loops can be effected via Henry's UNLOCKTHEINBOX.com website and will require a login to the site to complete.

10. Make certain ALL OUTGOING MESSAGES are AUTHENTICATED VIA SMTP.
     
    1. Do not send directly from a web form or newsletter generator.
        
    2. While it is OK to generate a message via a website, or generate a newsletter via a website, you MUST use SMTP AUTHENTICATION to SEND the generated message or sales receipt THROUCH AN MX SERVER using a VALID E-MAIL ADDRESS and PASSWORD from the DOMAIN which corresponds to the WEBSITE or other message generator because sending via an SMTP AUTHENTICATED account will properly created HEADERS in EVERY OUTBOUND MESSAGE!
        
11. Use rDNS, SPF, 2048 BIT DOMANKEYS, DKIM, and DMARC on ALL OUTGOING MESSAGES.  The first five are now REQUIRED by COMCAST and YAHOO!  DMARC is optional, but strongly suggested.
     
12. DMARC is a logical addition [see http://www.dmarc.org/ or see the DMARC information at http://www.unlocktheinbox.com/] because it truly locks down where your e-mail can be sent from and will literally block delivery of unauthorized messages originating from non-authorized MX servers and user accounts
    .
13. Do not omit the SPF and rDNS keys, most large ISPs, and many smaller ISPs as well, will never even accept your e-mail.  The receiving MX server will simply drop the connection and never allow the message to be received – without any notification to you.
     
14. Remember, GREYLISTING is now commonly used to help reduce spam.
     
    1. In order to make certain you are not contributing to Greylisting problems, make certain you RETRY messages which are not delivered the first attempt.
       
       ChicagoNetTech’s SmarterMail RETRY times are set as follows:
       
       [in minutes]: 5, 5, 15, 30, 30, 30, 30, 60, 90, 120, 240, 480, 960, 1440, 2880
        
    2. Trying more frequently during the first few attempts will probably result in your MX server being blocked by the receiving server.
        
    3. This retry schedule shown above will ensure that the IETF requirement of attempting to resend e-mail for UP TO FIVE DAYS is enforced.
        
    4. The number of retries is based on RFC 2821.4.5.4.1, which defines SMTP retry intervals and says, "Retries continue until the message is transmitted or the sender gives up; the give-up time generally needs to be at least 4-5 days."
        
    5. We allow messages to come through greylisting after 2 minutes and list the sender in our non-Greylisted database for 144 days after the initial successful delivery attempt.
        
    6. Many ISPs still have a 15 minute initial greylisting time established, so don't lose out on potential messages which are never delivered because of greylisting.

 

SUMMARY:  E-Mail is no longer just an “add-on” which is included with a webhosting package as a convenience for our customers.

 

E-Mail must be constantly monitored and a good ISP / MX operator is cognizant of what’s going on in his or her network and e-mail servers at all times.

 

Whether you’re confused or just need some help deciphering and implementing these new requirements, get in touch with us and we’ll be glad to help you out.

 

Just open a ticket with us and ChicagoNetTech will be glad to take a look at your SmarterMail or other e-mail server and help you sort things out.  If you don't already have an account on our Customer Service Portal, you'll need to create one prior to opening your new ticket.

 

Licensed SmarterMail software owners get a 35% discount over our standard rates when they require updates, troubleshooting, configuration, or any other SmarterMail server and/or maintenance assistance.

We can work on a case-by-case basis or on a monthly maintenance retainer with a level of support and/or service set by agreement with us and you.

Please setup an account and open a ticket or contact us via our contact form for more information on how we can turn your already great product into a minimum of maintenance and headaches.

 

Don’t forget to search our KB [knowledge base] for solutions to other problems and issues you might have encountered, too.

 

ChicagoNetTech's hosting and technical support rates are reasonable and we accept all major credit cards via PayPal.

For references and examples of my support of SmarterMail, please see my entries at: http://forums.smartertools.com/members/chicagonettech.7401/, where I have accumulated more than 3,550 support conversations since 2007.
 

Additional references upon request.

 

Copyright 2013 - 2014, ChicagoNetTech Inc, All Rights Reserved