Archive for the ‘Mail Servers’ Category

How a  Mail Server works

Introduction

images

 

This article is an introduction to Mail Server and it’s related protocols. Here I am explaining some basic ideas how a e-mail works and background information you need.

Electronic mail (email) is the term given to an electronic message, usually a form of simple text message, that a user types at a computer system and is transmitted over some form of computer network to another user, who can read it.

Email has become one of the driving forces behind connecting businesses to the Internet. It offers fast, economical transfer of messages anywhere in the world. Email messages doesn’t require paper or resources other than storage space thus we can support Green Environment policy.

Terminology

I think you heard about MUA, MTA and MDA while doing Linux training sessions. Do you know what is the exact role for these programs in a mail server? Here we are explaining in detail.

Mail User Agent (MUA) is mail client like Thunderbird,outlook, that allows a user to compose send and receive email.

Mail Transfer Agent (MTA) The MTA basically acts as a “mail router”. Which is responsible for sending the email to the recipient MTA. An MTA transfers mails via SMTP protocol. Later we explain in detail.

Mail Delivery Agent The recipient’s MTA receives the email and passes it on to a MDA. An MDA manages the user’s mailbox and handles the mails to deliver the MUA ie, (mail client) using either the POP3 or IMAP protocol. In other words, the MDA reads the header and sent back to a remote machine for email delivery.

How a Mail Server Works

We all have an email address. Do you ever think how an email works while we sending an e mail to some other recipient address. When we are clicking on the email send button, within minutes the email will deliver to the destination. But at first we have no idea, what are process take place behind the scene and to make sure that the email reaches it’s final destination. Here we are giving an introduction on various back end process of a Mail Server.

1 .When you compose an email with the help of Email Client, the email client will connect to your SMTP server the configuration that we have provided, while configuring your email client.

  1. For an example here the SMTP server is (mail.example.com). Once the email client has connected to mail.example.com SMTP server, it will forward the message that you have just composed to the server.
  2. An SMTP server is also called as MTA. Why we called as an MTA, because it’s works as a messenger. For example it transfers the mails and messages to the required recipients SMTP server.
  3. Next step is the SMTP server will do a DNS query for getting the SMTP server of your email receiver. And the SMTP server will try to find the domain SMTP server detail and handed over the message for the particular user. For example when you are sending an email to user@example.com. In that case the SMTP server will try to find the responsible domain SMTP server detail, example.com and hand over the message for the user.
  4. Once your SMTP server finds the target receivers SMTP server, using an MX record lookup, it will forward the message to that server. Then the SMTP server will forward the message to POP3 or IMAP server responsible for the domain.

Mail server functionality can be divided into two processes:

Sending and Receiving emails

Sending email : Simple Mail Transfer Protocol (SMTP)
Receiving email : Post Office Protocol (POP3) / Internet Message Access Protocol (IMAP)

What is an SMTP protocol and why we are using this protocol in mail servers?

SMTP (Simple Mail Transfer Protocol) is a TCP/IP protocol used in sending and receiving e-mail. In simply the purpose of SMTP is to set up communication rules between servers, like (postman). When we are sending a mail to some other recipient address, the connection happens via SMTP protocol.

Comparison between IMAP and POP3 protocol

IMAP and POP3 are two different protocols

The main difference is that IMAP(Internet Messaged Access Protocol) always syncs with mail server. So whenever you made any changes on your mail client, the result will instantly appears on your inbox.

The biggest advantage of using IMAP is you can access your mail from multiple mail clients like Thunderbird and each mail client will shows real-time update. Suppose mail server is connected with two different mail clients on different computers. If one of the user deletes a message from mail client, the changes will immediately update on both mail server and client.

This why IMAP is more suitable and if you’re going to access your emails from different locations or if your mails are managed by multiple users.

In POP3 (Post Office Protocol), your mail client (Thunderbird) and mail server will not synced. This means the mails are downloaded in your computer and the changes won’t reflect on the server.

In POP3 multiple mail clients access were not supported. Here you can download emails from mail server to your computer using POP3. After downloading, the original mails were removed from the server. Hence you can’t access the mails from another computer.

Mail Service protocols and ports

Simple Mail Transfer Protocol (SMTP)

Port 25 – SMTP non-encrypted port
Port 465 – SMTP secure port

Post Office Protocol version 3 (POP3)

Port 110 – POP3 non-encrypted port
Port 995 – POP3 secure port

Internet Message Access Protocol (IMAP)

Port 143 – IMAP non-encrypted port
Port 993 – IMAP secure port

MX Record & priorities

Mail exchange is just another name for the machine whose primary function is receiving and sending email. An MX record is set to point a canonical hostname, like example.com or mail.example.com. You can find your domain mail server detail either from dig or online tools, like intodns.com and mxtoolbox.com

The MX record uses preference values to specify the routing order –low value = high priority. The MX priority determines the order (which mail server) that your mail will be attempted to be delivered. The mail server with the lowest MX priority will first be attempted to deliver.

google.com. 600 IN MX 40 alt3.aspmx.l.google.com.
google.com. 600 IN MX 30 alt2.aspmx.l.google.com.
google.com. 600 IN MX 50 alt4.aspmx.l.google.com.
google.com. 600 IN MX 10 aspmx.l.google.com.
google.com. 600 IN MX 20 alt1.aspmx.l.google.com.
So, if you have five MX records with levels 10, 20, 30, 40 ,50 the following would occur. Mail would always be first tried to be delivered to the MX record with MX priority of 10. If that mail server is down then the mail will try to be delivered to the mail server at 20. If the mail server at priority 20 is down then the mail will be attempted to be delivered at the mail server at priority 30. If you have multiple MX records with the same MX priority, then it will setup a round robin configuration for your email.

Types of Mail Service

Dovecot
Courier

In cPanel servers we are using dovecot and courier mail services. This is the service which delivers the email to your inbox while exim is the one which sends the mails to the servers.

Dovecot

Dovecot uses less memory,better performance and is more configurable. Dovecot is a POP3 and IMAP mail server that can work with standard mbox and maildir formats. Dovecot is much faster than Courier and the advantage is its intelligent use of configuration files.

Files

• Program : /usr/sbin/dovecot
• Init Script : /etc/init.d/dovecot
• Config : /etc/dovecot.conf

Courier

Courier is extremely reliable, but it needs a larger memory. Why it takes heavy memory usage, because when ever you open web mail client it has to reload the entire inbox. Which makes it a bit slower to load compared to dovecot, when we are dealing with large mailboxes.

Files

• Program : /usr/lib/courier-imap/libexec/couriertcpd
• Init Script : /etc/init.d/courier
• Config : /usr/local/libexec/courier-imap

If you are running a shared server, dovecot need to deliver better performance while you have a dedicated server for one or two websites, courier can be more efficient. The default choice for cPanel is the dovecot and if you need to change you need to navigate to WHM Main >> Service Configuration >> Mailserver Selection

OR

You can change it from back-end command line using cPanel script and editing config file.

/scripts/setupmailserver dovecot

Then set /var/cpanel/cpanel.config

maildir=1
mailserver=dovecot
Exim Configuration files

Exim is an open source mail transfer agent (MTA), which is a program responsible for Receiving, Routing, Delivering e-mail messages

Exim commands

exim -bp: shows messages in queue
exim -bpc : shows the no.of messages in queue
exiwhat : Finding out what Exim processes are doing
exim -qff : Attempt to flush frozen messages
exim -bp | exiqsumm : Print a summary of messages in the queue

exiqgrep usage

exiqgrep -f [luser]@domain : Search the queue for messages from a specific sender
exiqgrep -r [luser]@domain : Search the queue for messages for a specific recipient/domain
exiqgrep -i : To Print just the message-id of the entire queue
exim -Mrm: <message-id> : Remove a message from the queue
exim -M <message-id>: Deliver a specific message
exiqgrep -z -i | xargs exim -Mrm: Remove all frozen messages

Files

/etc/exim.conf – mail server configuration file
/etc/localdomains – exim related file. All the domains using the same server’s MX, should be listed here to be able to send/receive emails.

/etc/valiases/domainname – catchall and forwarders are set here.
/etc/vfilters/domainname – email filters are set here.
/var/spool/exim/input – Mail queue.
/var/spool/exim/msglog – email message logs.
/var/cpanel/horde – version file, backups and logs stored here
/var/cpanel/roundcube – version file, backups and logs stored here
/var/cpanel/squirrelmail – version file stored here.

You can also edit and modify exim configuration from WHM

Home >> Service Configuration >> Exim Configuration Manager >> Advanced Editor

How to change exim Mail server IP address

When ever our main server is blacklisted in RBL sites. This is only a temporary work around to the blacklist problem and you have to make sure that you have identify the spammer and resolved the issue. As a temporary solution we can change the mail server IP address, this resolve the email bounce back issue. If you have any email script under the domains, then you need to change the outgoing mail IP address on that script. In some cases PHP mailer script will give you bounce back message after changing the mail server IP address.

Changing the IP Globally

You need edit the following file /etc/mailips

*: xxx.xxx.xxx.xxx

Then add the IP and it’s matching PTR to /etc/mail_reverse_dns:

xxx.xxx.xxx.xxx hostname.tld

This will tell Exim to use that IP for any sender on the server.

Spamassin

Apache SpamAssassin is an email utility that examines incoming email and tests for spam characteristics. SpamAssassin is designed to identify and mark e-mails that score beyond your threshold value. SpamAssassin has 10 different levels of settings to catch spam. By default the spam score will be 5. You can enable the SpamAssassin from the domain cPanel itself.

How spam score works

If you set lower the score, more email will be caught as spam. For example, you have enabled spam score as “1” that means only one hit needs to be flagged against the email to be considered as spam. If you set the spam score higher, more hits will be required on an email for it to be labeled as spam. So if we set lower spam score, more emails should be flagged as spam.

0 means everything incoming will be marked as spam.
5 is the default setting (and works well for typical users).
10 means that any message with a score of 10 or less will not be marked as spam.
RBL

What is an RBL ?

A DNS-based Blackhole List or Real-time Blackhole List is a list of IP addresses which are most often used to publish the addresses of computers or networks linked to spamming, most mail server software can be configured to reject or flag messages which have been sent from a site listed on one or more such lists.

Eg: spamhaus,Spamcop etc

How to check whether an IP is blacklisted ?

You can check whether the server mail IP address is blacklisted in any RBL’s using the below link

Eg: http://mxtoolbox.com/blacklists.aspx

How to delist the IP address

For delisting you need to contact the blacklist provider from there you can check whether the IP address was listed on their database. eg, http://www.spamhaus.org/lookup/ .It will take 48 hours to complete the delist process.

How we can prevent spamming activities and what are the methods ?

We all know about spam mails right? We are getting so many spam mail in our personal mails, but in Gmail,yahoo and other service providers have their own techniques to prevent spam mails. Here I am sharing a brief description about two different technologies in cPanel SPF and DKIM.

Whenever you create a domain on CPanel server using WHM, it won’t add domainkeys and SPF records for particular domains. We need to add it manually.

Sender Policy Framework (SPF)

SPF helps prevent spammers, SPF can also reduce the amount of bounce messages that you receive. SPF uses DNS records that specify the mail servers and IP addresses that are authorized to send e-mail messages from a domain.

To Enable SPF Records for a domain

Log into cPanel
Click Email Authentication
Click ENABLE beside the SPF section.
Click the Go Back link
Verify that the Status now says that it is Active and Enabled

You can add from command prompt too.

/usr/local/cpanel/bin/domain_keys_installer “domain username”

DomainKeys Identified Mail (DKIM)

You can use DKIM to verify an incoming e-mail message is actually from the stated sender, and that the message has not been altered during transit. When DKIM is enabled, the sender digitally signs a message using a private key. The recipient uses DNS to retrieve the sender’s public key and verify the message’s signature. If the signature is invalid, then the message is assumed to be forged and therefore spam.

To Enable Domain Keys

Log into cPanel
Click Email Authentication
Click ENABLE beside the Domain Keys section
Click the Go Back link
Verify that the Status now says that it is Active and Enabled

You can add from command prompt too.

/usr/local/cpanel/bin/domain_keys_installer “domain user name”

How to check e-mail log using exigrep

Here I am explaining how to trace and identify the mail log symbols. One of the best tools you can use when tracking down e-mail problems is mail logs.

First, it is a good idea to get to know the following symbols.

<=
=>
->
*>
**

At least one of these symbols will be on nearly every line of exim’s mainlog. Other abbreviations in the log will change their meaning based on which of these symbols is on that line. These abbreviations consist of one or two letters.

2014-10-07 03:41:47 [4578] 1XbPOr-0001Bq-OC H=ccm181.constantcontact.com [208.75.123.181]:58477 I=[72.34.47.10]:25 Warning: “SpamAssassin as ahmedkha detected message as NOT spam (-4.6)”

2014-10-07 03:41:47 [4578] 1XbPOr-0001Bq-OC <= A2tcbHRYaQHK0cJzWtAZ5dg==_1110613794467_pL+XcLVEEeOkRNSuUnVJUA==@in.constantcontact.com H=ccm181.constantcontact.com [208.75.123.181]:58477 I=[72.34.47.10]:25 P=esmtp S=37537 M8S=0 id=1118748363623.1110613794467.22409.0.210333JL.1002@scheduler.constantcontact.com T=”Ne ratez pas le RV incontournable du business de la television en\n Afrique !” from <A2tcbHRYaQHK0cJzWtAZ5dg==_1110613794467_pL+XcLVEEeOkRNSuUnVJUA==@in.constantcontact.com> for ahmed@ahmedkhaled.com

Beginning of the Line

Each line starts with the date and time. Immediately following is exim’s internal message ID:
First Line

The next item on the first line starts with “H=”. This specifies the host name of the server that the mail originates from. Next (in square brackets) is the IP address of that server, followed by the port number. “I=” looks similar, but is describing your mail server. A warning follows: Spamassassin believes that this message is not spam.

Second Line

The second line almost starts the same (with H and I), but before that the “<=” symbol is used. This indicates that this line is describing the arrival of the message on your mail server. Immediately after this symbol is the e-mail address that this mail is being sent from.

After H and I, the “P” abbreviation designates the protocol being used (here esmtps). Note that this only means this because it is on a “<=” line; if this were a “=>” line it would indicate the return path of the message.

The X tells us the particular cipher suite that is used; this is usually not something you would be looking for when troubleshooting a mail issue. CV refers to certification verification status, and S is the size of the message.

Next, “id” refers to the message id that was created by the sending server and sent as one of the mail headers. T (for topic) is the subject of the e-mail. The line ends with “for user@example.com”, letting us know just who the message is for.

Steps to troubleshooting spamming

https://linuxtechme.wordpress.com/2012/04/23/spamming-in-exim/

Mail Server

Advertisements

If you are unable to login to your email accounts and the system shows this error message:

Plaintext authentication disallowed on non-secure (SSL/TLS) connections

If you don’t want to use SSL/TLS connection to get your email, and to disable SSL/TLS secure connection.

You can verify this error using telnet as follows.

[ryan@ ~]$ telnet mail.domain.com 110
Trying 192.40.112.75…
Connected to mail.domain.com.
Escape character is ‘^]’.
+OK Dovecot ready.
user ryan@domain.com
-ERR [AUTH] Plaintext authentication disallowed on non-secure (SSL/TLS) connections.

>> Edit dovecot configuration file: /etc/dovecot.conf

>> Change the value for this directive from:

disable_plaintext_auth = yes
TO:
disable_plaintext_auth = no

Restart service dovecot and exim
# /etc/init.d/dovecot restart
# /etc/init.d/exim restart

If these steps are not working check cPanel logs.

Error :-
PHP Fatal error: Call to undefined function logout_error() in /usr/local/cpanel/base/3rdparty/squirrelmail/plugins/login_auth/functions.php on line 282

Check the file permission

# ls -l /usr/local/cpanel/bin/dovecot-wrap
-rwsr-x— 1 root dovecot 9176 Apr 9 2013 /usr/local/cpanel/bin/dovecot-wrap*

chmod +x dovecot-wrap

In my case the issue was unresolved, so I ran an upcp that fixed the issue.

[ryan@ ~]$ telnet mail.domain.com 110
Trying 192.40.112.75…
Connected to mail.domain.com
Escape character is ‘^]’.
+OK Dovecot ready.
user ryan@domain.com
+OK

=-=-=-=-=-=-=-=-=

We are getting connection refused error while trying to access the server via pop. Please check the below logs.

26 01:57:18 host2 dovecot: pop3-login: Maximum number of connections from user+IP exceeded (mail_max_userip_connections=3): user=, method=PLAIN, rip=112.29.18.1, lip=69.22.10.17

Solution :-

Edit /etc/dovecot.conf and check the parameter “mail_max_userip_connections” was set to 3.

mail_max_userip_connections = 3

Increased it to a higher value and the issue got fixed.

Bounced Message “retry time not reached for any host after a long failure period”

It is likely due to an error with the server sending mail at their end. The Exim Retry Database has likely become corrupt and would need to be fixed before they could retry sending mail to your email address/domain.

cd /var/spool/exim/db
rm -f retry retry.lockfile
rm -f wait-remote_smtp wait-remote_smtp.lockfile
service exim restart

Or, use the following commands:
/usr/sbin/exim_tidydb -t 1d /var/spool/exim retry > /dev/null
/usr/sbin/exim_tidydb -t 1d /var/spool/exim reject > /dev/null
/usr/sbin/exim_tidydb -t 1d /var/spool/exim wait-remote_smtp > /dev/null
/scripts/courierup — force
/scripts/eximup –force

Connection dropped by IMAP server : When you are trying to login to email account in Roundcube Webmail client.

You may fix the issue by follow the steps below:

If you are using Roundecube check the following logs :

# tail -50 /var/cpanel/roundcube/log/errors

1. Try to restart the IMAP service

# /etc/init.d/courier-imap restart

2. Try to update the courier imap server

# /scripts/courierup –force

3. If the issue still not solve, run the command below to fix the permission for mail directory

chown -R USERNAME:mail /home/USERNAME/mail/
chown -R USERNAME:mail /homeUSERNAME//etc/

Where USERNAME is his cPanel username

How to manage old emails using CPanel

ConfigServer Mail Manage (cmm)

It provides an interface to the cPanel user accounts email configuration without having to login to their accounts. It is domain based rather than account based and allows you to do all the following from within WHM:

To install or upgrade cmm simply do the following from the root shell via SSH:

cd /usr/src
rm -fv /usr/src/cmm.tgz
wget http://www.configserver.com/free/cmm.tgz
tar -xzf cmm.tgz
cd cmm
sh install.sh
rm -Rfv /usr/src/cmm*

Then login to WHM and scroll to the bottom of the left hand menu and you should see “ConfigServer Mail Manage”

If you want to uninstall, simply:

cd /usr/src
rm -fv /usr/src/cmm.tgz
wget http://www.configserver.com/free/cmm.tgz
tar -xzf cmm.tgz
cd cmm
sh uninstall.sh
rm -Rfv /usr/src/cmm*

ConfigServer Mail Queues (cmq)

Integrates with MailScanner, if installed, to offer views/deletion of email in both the Pending and Delivery queues. Searching for and viewing/deleting emails to/from specific domains and addresses

cd /usr/src
wget http://www.configserver.com/free/cmq.tgz
tar -xzf cmq.tgz
cd cmq
sh install.sh

Login to WHM and scroll to the bottom of the left hand menu and you should see “ConfigServer Mail Queues”

If you want to uninstall, simply:

cd /usr/src
rm -fv /usr/src/cmq.tgz
wget http://www.configserver.com/free/cmq.tgz
tar -xzf cmq.tgz
cd cmq
sh uninstall.sh
rm -Rfv /usr/src/cmq*

First of all, login to your cPanel account and scroll down to the “Email Management Tools” box and enable SpamAssassin

Click on SpamAssassin and check if it’s active. If not, activate it.

You can configure it by clicking on the “Configure SpamAssassin” button and set the score to 5 or 6. The greater is the number and more possible SPAM you could receive, so consider staying on 5 or 6, as a recommendation. Remember to save.

You can also configure Mozilla Thunderbird to accept SpamAssasin’s headers to help it identify SPAM. Set it on your account settings on a “per account” basis.

Set filters in “Account Level Filtering”

You can add the filter for a specific email address from “cPanel>> Mail>> User Level Filtering” option to DISCARD the incoming emails from that malicious email address you want to block. Here are the steps to set a filter.

1. login to cPanel and go to “cPanel>> Mail>> User Level Filtering” option.

2. Add the following filter for your domain email accounts.

Rules:

Select “From” from drop down menu, then you’ll need to select “Contains” from the drop down menu.

Actions:

Enter the email address you want to block” and click on Activate button.

This filter will block one particular email address from coming into the email address on your domain email address\es and at the same time, your email address will not send any bounce back email message to the blocked email address.
———————————————>

In your Exim logs you may see errors like this: Warning: ACL “warn” statement skipped: condition test deferred: ratelimit database not available

temporarily rejected connection in ?connect? ACL: ratelimit database not available

This likely means the Exim cache databases are corrupted. You can clear them by running:

# rm -fv /var/spool/exim/db/*

Then restart Exim.

The files in question are:

/var/spool/exim/db/ratelimit
/var/spool/exim/db/ratelimit.lockfile
/var/spool/exim/db/retry
/var/spool/exim/db/retry.lockfile
/var/spool/exim/db/wait-dk_remote_smtp
/var/spool/exim/db/wait-dk_remote_smtp.lockfile
/var/spool/exim/db/wait-remote_smtp
/var/spool/exim/db/wait-remote_smtp.lockfile

Unable to login the Rouncube. Make sure the MySQL password is right

root@server[~]$ grep db_dsnw /usr/local/cpanel/base/3rdparty/roundcube/config/db.inc.php
$rcmail_config[‘db_dsnw’] = ‘mysql://roundcube:ASm_FJDYv2CGfEtH@localhost/roundcube’;

The password in this case is set in the config as: “ASm_FJDYv2CGfEtH”. See if the hash matches what is set for the user:

root@server[~]$ mysql
mysql> use mysql;
mysql> select PASSWORD(‘ASm_FJDYv2CGfEtH’);
mysql> select * from user where User = ’roundcube’;
mysql> update user set Password = PASSWORD(‘ASm_FJDYv2CGfEtH’) where User = ’roundcube’;

If the above didn’t fix it, try to update RoundCube (the right way)

root@server[~]$ mysqldump roundcube > /path/to/backup.sql
root@server[~$ /usr/local/cpanel/bin/update-roundcube –force
root@server[~]$ mysql roundcube < /path/to/backup.sql

If it is not working, please go through these steps

mysql -e “show table status” roundcube |grep -i innodb |awk ‘{print $1,$2}’
cache InnoDB
cache_index InnoDB
cache_messages InnoDB
cache_thread InnoDB
contactgroupmembers InnoDB
contactgroups InnoDB
contacts InnoDB
dictionary InnoDB
identities InnoDB
searches InnoDB
session InnoDB
users InnoDB

Somehow InnoDB had become corrupted and we resolved it by adding

innodb_force_recovery=6

to my.cnf and starting MySQL. Everything was working fine except Roundcube. Commenting out this line and restarting MySQL allowed Roundcube to begin working again.

ERROR:Message not sent. Server replied: Service not available, closing channel
421 Too many concurrent SMTP connections; please try again later.

It means that there are to many open SMTP connections either incoming or
outgoing.The default is 20, you can raise that with the smtp_accept_max
command in your configuration file.

you can check connections through command:

cat /etc/exim.conf |grep smtp_accept_max

By default ,the value is 100 ,You can raise it to 150 then run :

Service exim restart

You can get all the IPs addresses connected to/from TCP port 25 at any given time by running the following command :

netstat -plan|grep ‘:25′|grep ESTAB

That will show the IP addresses connected to/from TCP port 25 at any given time. It’ll change by the second. But if you have 250+ connections and you think there is likely tons of connections from or to single source, you’ll be able to see that easily. Then figure out why that IP address is connecting so much.

grep xxx.xxx.xxx.xxx /var/log/exim_mainlog

# ps -auxwwf | grep exim

# netstat -ano | grep “25”