How a Mail Server works
Introduction
This article is an introduction to Mail Server and it’s related protocols. Here I am explaining some basic ideas how a e-mail works and background information you need.
Electronic mail (email) is the term given to an electronic message, usually a form of simple text message, that a user types at a computer system and is transmitted over some form of computer network to another user, who can read it.
Email has become one of the driving forces behind connecting businesses to the Internet. It offers fast, economical transfer of messages anywhere in the world. Email messages doesn’t require paper or resources other than storage space thus we can support Green Environment policy.
Terminology
I think you heard about MUA, MTA and MDA while doing Linux training sessions. Do you know what is the exact role for these programs in a mail server? Here we are explaining in detail.
Mail User Agent (MUA) is mail client like Thunderbird,outlook, that allows a user to compose send and receive email.
Mail Transfer Agent (MTA) The MTA basically acts as a “mail router”. Which is responsible for sending the email to the recipient MTA. An MTA transfers mails via SMTP protocol. Later we explain in detail.
Mail Delivery Agent The recipient’s MTA receives the email and passes it on to a MDA. An MDA manages the user’s mailbox and handles the mails to deliver the MUA ie, (mail client) using either the POP3 or IMAP protocol. In other words, the MDA reads the header and sent back to a remote machine for email delivery.
How a Mail Server Works
We all have an email address. Do you ever think how an email works while we sending an e mail to some other recipient address. When we are clicking on the email send button, within minutes the email will deliver to the destination. But at first we have no idea, what are process take place behind the scene and to make sure that the email reaches it’s final destination. Here we are giving an introduction on various back end process of a Mail Server.
1 .When you compose an email with the help of Email Client, the email client will connect to your SMTP server the configuration that we have provided, while configuring your email client.
- For an example here the SMTP server is (mail.example.com). Once the email client has connected to mail.example.com SMTP server, it will forward the message that you have just composed to the server.
- An SMTP server is also called as MTA. Why we called as an MTA, because it’s works as a messenger. For example it transfers the mails and messages to the required recipients SMTP server.
- Next step is the SMTP server will do a DNS query for getting the SMTP server of your email receiver. And the SMTP server will try to find the domain SMTP server detail and handed over the message for the particular user. For example when you are sending an email to user@example.com. In that case the SMTP server will try to find the responsible domain SMTP server detail, example.com and hand over the message for the user.
- Once your SMTP server finds the target receivers SMTP server, using an MX record lookup, it will forward the message to that server. Then the SMTP server will forward the message to POP3 or IMAP server responsible for the domain.
Mail server functionality can be divided into two processes:
Sending and Receiving emails
Sending email : Simple Mail Transfer Protocol (SMTP)
Receiving email : Post Office Protocol (POP3) / Internet Message Access Protocol (IMAP)
What is an SMTP protocol and why we are using this protocol in mail servers?
SMTP (Simple Mail Transfer Protocol) is a TCP/IP protocol used in sending and receiving e-mail. In simply the purpose of SMTP is to set up communication rules between servers, like (postman). When we are sending a mail to some other recipient address, the connection happens via SMTP protocol.
Comparison between IMAP and POP3 protocol
IMAP and POP3 are two different protocols
The main difference is that IMAP(Internet Messaged Access Protocol) always syncs with mail server. So whenever you made any changes on your mail client, the result will instantly appears on your inbox.
The biggest advantage of using IMAP is you can access your mail from multiple mail clients like Thunderbird and each mail client will shows real-time update. Suppose mail server is connected with two different mail clients on different computers. If one of the user deletes a message from mail client, the changes will immediately update on both mail server and client.
This why IMAP is more suitable and if you’re going to access your emails from different locations or if your mails are managed by multiple users.
In POP3 (Post Office Protocol), your mail client (Thunderbird) and mail server will not synced. This means the mails are downloaded in your computer and the changes won’t reflect on the server.
In POP3 multiple mail clients access were not supported. Here you can download emails from mail server to your computer using POP3. After downloading, the original mails were removed from the server. Hence you can’t access the mails from another computer.
Mail Service protocols and ports
Simple Mail Transfer Protocol (SMTP)
Port 25 – SMTP non-encrypted port
Port 465 – SMTP secure port
Post Office Protocol version 3 (POP3)
Port 110 – POP3 non-encrypted port
Port 995 – POP3 secure port
Internet Message Access Protocol (IMAP)
Port 143 – IMAP non-encrypted port
Port 993 – IMAP secure port
MX Record & priorities
Mail exchange is just another name for the machine whose primary function is receiving and sending email. An MX record is set to point a canonical hostname, like example.com or mail.example.com. You can find your domain mail server detail either from dig or online tools, like intodns.com and mxtoolbox.com
The MX record uses preference values to specify the routing order –low value = high priority. The MX priority determines the order (which mail server) that your mail will be attempted to be delivered. The mail server with the lowest MX priority will first be attempted to deliver.
google.com. 600 IN MX 40 alt3.aspmx.l.google.com.
google.com. 600 IN MX 30 alt2.aspmx.l.google.com.
google.com. 600 IN MX 50 alt4.aspmx.l.google.com.
google.com. 600 IN MX 10 aspmx.l.google.com.
google.com. 600 IN MX 20 alt1.aspmx.l.google.com.
So, if you have five MX records with levels 10, 20, 30, 40 ,50 the following would occur. Mail would always be first tried to be delivered to the MX record with MX priority of 10. If that mail server is down then the mail will try to be delivered to the mail server at 20. If the mail server at priority 20 is down then the mail will be attempted to be delivered at the mail server at priority 30. If you have multiple MX records with the same MX priority, then it will setup a round robin configuration for your email.
Types of Mail Service
Dovecot
Courier
In cPanel servers we are using dovecot and courier mail services. This is the service which delivers the email to your inbox while exim is the one which sends the mails to the servers.
Dovecot
Dovecot uses less memory,better performance and is more configurable. Dovecot is a POP3 and IMAP mail server that can work with standard mbox and maildir formats. Dovecot is much faster than Courier and the advantage is its intelligent use of configuration files.
Files
• Program : /usr/sbin/dovecot
• Init Script : /etc/init.d/dovecot
• Config : /etc/dovecot.conf
Courier
Courier is extremely reliable, but it needs a larger memory. Why it takes heavy memory usage, because when ever you open web mail client it has to reload the entire inbox. Which makes it a bit slower to load compared to dovecot, when we are dealing with large mailboxes.
Files
• Program : /usr/lib/courier-imap/libexec/couriertcpd
• Init Script : /etc/init.d/courier
• Config : /usr/local/libexec/courier-imap
If you are running a shared server, dovecot need to deliver better performance while you have a dedicated server for one or two websites, courier can be more efficient. The default choice for cPanel is the dovecot and if you need to change you need to navigate to WHM Main >> Service Configuration >> Mailserver Selection
OR
You can change it from back-end command line using cPanel script and editing config file.
/scripts/setupmailserver dovecot
Then set /var/cpanel/cpanel.config
maildir=1
mailserver=dovecot
Exim Configuration files
Exim is an open source mail transfer agent (MTA), which is a program responsible for Receiving, Routing, Delivering e-mail messages
Exim commands
exim -bp: shows messages in queue
exim -bpc : shows the no.of messages in queue
exiwhat : Finding out what Exim processes are doing
exim -qff : Attempt to flush frozen messages
exim -bp | exiqsumm : Print a summary of messages in the queue
exiqgrep usage
exiqgrep -f [luser]@domain : Search the queue for messages from a specific sender
exiqgrep -r [luser]@domain : Search the queue for messages for a specific recipient/domain
exiqgrep -i : To Print just the message-id of the entire queue
exim -Mrm: <message-id> : Remove a message from the queue
exim -M <message-id>: Deliver a specific message
exiqgrep -z -i | xargs exim -Mrm: Remove all frozen messages
Files
/etc/exim.conf – mail server configuration file
/etc/localdomains – exim related file. All the domains using the same server’s MX, should be listed here to be able to send/receive emails.
/etc/valiases/domainname – catchall and forwarders are set here.
/etc/vfilters/domainname – email filters are set here.
/var/spool/exim/input – Mail queue.
/var/spool/exim/msglog – email message logs.
/var/cpanel/horde – version file, backups and logs stored here
/var/cpanel/roundcube – version file, backups and logs stored here
/var/cpanel/squirrelmail – version file stored here.
You can also edit and modify exim configuration from WHM
Home >> Service Configuration >> Exim Configuration Manager >> Advanced Editor
How to change exim Mail server IP address
When ever our main server is blacklisted in RBL sites. This is only a temporary work around to the blacklist problem and you have to make sure that you have identify the spammer and resolved the issue. As a temporary solution we can change the mail server IP address, this resolve the email bounce back issue. If you have any email script under the domains, then you need to change the outgoing mail IP address on that script. In some cases PHP mailer script will give you bounce back message after changing the mail server IP address.
Changing the IP Globally
You need edit the following file /etc/mailips
*: xxx.xxx.xxx.xxx
Then add the IP and it’s matching PTR to /etc/mail_reverse_dns:
xxx.xxx.xxx.xxx hostname.tld
This will tell Exim to use that IP for any sender on the server.
Spamassin
Apache SpamAssassin is an email utility that examines incoming email and tests for spam characteristics. SpamAssassin is designed to identify and mark e-mails that score beyond your threshold value. SpamAssassin has 10 different levels of settings to catch spam. By default the spam score will be 5. You can enable the SpamAssassin from the domain cPanel itself.
How spam score works
If you set lower the score, more email will be caught as spam. For example, you have enabled spam score as “1” that means only one hit needs to be flagged against the email to be considered as spam. If you set the spam score higher, more hits will be required on an email for it to be labeled as spam. So if we set lower spam score, more emails should be flagged as spam.
0 means everything incoming will be marked as spam.
5 is the default setting (and works well for typical users).
10 means that any message with a score of 10 or less will not be marked as spam.
RBL
What is an RBL ?
A DNS-based Blackhole List or Real-time Blackhole List is a list of IP addresses which are most often used to publish the addresses of computers or networks linked to spamming, most mail server software can be configured to reject or flag messages which have been sent from a site listed on one or more such lists.
Eg: spamhaus,Spamcop etc
How to check whether an IP is blacklisted ?
You can check whether the server mail IP address is blacklisted in any RBL’s using the below link
Eg: http://mxtoolbox.com/blacklists.aspx
How to delist the IP address
For delisting you need to contact the blacklist provider from there you can check whether the IP address was listed on their database. eg, http://www.spamhaus.org/lookup/ .It will take 48 hours to complete the delist process.
How we can prevent spamming activities and what are the methods ?
We all know about spam mails right? We are getting so many spam mail in our personal mails, but in Gmail,yahoo and other service providers have their own techniques to prevent spam mails. Here I am sharing a brief description about two different technologies in cPanel SPF and DKIM.
Whenever you create a domain on CPanel server using WHM, it won’t add domainkeys and SPF records for particular domains. We need to add it manually.
Sender Policy Framework (SPF)
SPF helps prevent spammers, SPF can also reduce the amount of bounce messages that you receive. SPF uses DNS records that specify the mail servers and IP addresses that are authorized to send e-mail messages from a domain.
To Enable SPF Records for a domain
Log into cPanel
Click Email Authentication
Click ENABLE beside the SPF section.
Click the Go Back link
Verify that the Status now says that it is Active and Enabled
You can add from command prompt too.
/usr/local/cpanel/bin/domain_keys_installer “domain username”
DomainKeys Identified Mail (DKIM)
You can use DKIM to verify an incoming e-mail message is actually from the stated sender, and that the message has not been altered during transit. When DKIM is enabled, the sender digitally signs a message using a private key. The recipient uses DNS to retrieve the sender’s public key and verify the message’s signature. If the signature is invalid, then the message is assumed to be forged and therefore spam.
To Enable Domain Keys
Log into cPanel
Click Email Authentication
Click ENABLE beside the Domain Keys section
Click the Go Back link
Verify that the Status now says that it is Active and Enabled
You can add from command prompt too.
/usr/local/cpanel/bin/domain_keys_installer “domain user name”
How to check e-mail log using exigrep
Here I am explaining how to trace and identify the mail log symbols. One of the best tools you can use when tracking down e-mail problems is mail logs.
First, it is a good idea to get to know the following symbols.
<=
=>
->
*>
**
At least one of these symbols will be on nearly every line of exim’s mainlog. Other abbreviations in the log will change their meaning based on which of these symbols is on that line. These abbreviations consist of one or two letters.
2014-10-07 03:41:47 [4578] 1XbPOr-0001Bq-OC H=ccm181.constantcontact.com [208.75.123.181]:58477 I=[72.34.47.10]:25 Warning: “SpamAssassin as ahmedkha detected message as NOT spam (-4.6)”
2014-10-07 03:41:47 [4578] 1XbPOr-0001Bq-OC <= A2tcbHRYaQHK0cJzWtAZ5dg==_1110613794467_pL+XcLVEEeOkRNSuUnVJUA==@in.constantcontact.com H=ccm181.constantcontact.com [208.75.123.181]:58477 I=[72.34.47.10]:25 P=esmtp S=37537 M8S=0 id=1118748363623.1110613794467.22409.0.210333JL.1002@scheduler.constantcontact.com T=”Ne ratez pas le RV incontournable du business de la television en\n Afrique !” from <A2tcbHRYaQHK0cJzWtAZ5dg==_1110613794467_pL+XcLVEEeOkRNSuUnVJUA==@in.constantcontact.com> for ahmed@ahmedkhaled.com
Beginning of the Line
Each line starts with the date and time. Immediately following is exim’s internal message ID:
First Line
The next item on the first line starts with “H=”. This specifies the host name of the server that the mail originates from. Next (in square brackets) is the IP address of that server, followed by the port number. “I=” looks similar, but is describing your mail server. A warning follows: Spamassassin believes that this message is not spam.
Second Line
The second line almost starts the same (with H and I), but before that the “<=” symbol is used. This indicates that this line is describing the arrival of the message on your mail server. Immediately after this symbol is the e-mail address that this mail is being sent from.
After H and I, the “P” abbreviation designates the protocol being used (here esmtps). Note that this only means this because it is on a “<=” line; if this were a “=>” line it would indicate the return path of the message.
The X tells us the particular cipher suite that is used; this is usually not something you would be looking for when troubleshooting a mail issue. CV refers to certification verification status, and S is the size of the message.
Next, “id” refers to the message id that was created by the sending server and sent as one of the mail headers. T (for topic) is the subject of the e-mail. The line ends with “for user@example.com”, letting us know just who the message is for.
Steps to troubleshooting spamming
https://linuxtechme.wordpress.com/2012/04/23/spamming-in-exim/
Linux System Administration 