Load Tackling in cPanel Servers

We all know that a server’s stability is dependent on its load, directly. Load, in computing, is a measure of the amount of processing a computer is currently performing, usually as some variation on a percentage. Load spike is something from which you would always want your servers to stay away from.
This article mainly focus on finding the probable causes for a high load. I hope this article will help you in fighting those load spike nightmares.

Load Checking Commands
1.w
2.uptime
3.pstree
4.top
5.mysqladmin proc stat

Causes for Load

High resource usage by some processes
Perl script attacks
Spam Attack
Vulnerable PHP scripts
MySQL Abuse
DOS attack

High Resource Usage by Processes
You can find the resource usage, by the command ‘top -c’ and ‘pstree -apu’. Install CSF and configure it to get alerts on “Excessive resource usage.”
To get the email alerts, you can set your email address in this file: /etc/csf/resalert.txt

Perl Script Attacks
These attacks are usually caused by poor coding or vulnerabilities in the software version. If Perl scripts are running in the server, execute ‘pstree -apu’. You can see something similar to the snippet given below.
├─perl,30478,user
│ └─sh,30479 -c echo 40″`uname 40-a`”;echo 40″`id`”;/bin/sh
│ └─sh,30485
To find the source of the Perl script, execute ls -alh /proc/30478 |grep cwd
lrwxrwxrwx 1 user user 0 Sep 28 06:16
cwd -> /home/user/public_html/media/
This means that, the Perl script is running from the location, /home/user/public_html/media/.
Also, you can use the command “ lsof -p 30478 |less ” to get more information.

Preventive measures for blocking such Perl attacks include:
Disable allow_url_fopen and allow_url_include in your php.ini (/usr/local/lib/php.ini) file
Disable the following functions in PHP:
disable_functions = exec, shellexec, passthru, system, escapeshellcmd, escapeshellarg
Install mod_security (this also helps in reducing Iframe attacks )

Spamming
Spamming is basically flooding a server with multiple copies of the same message. This can be either incoming or outgoing.
Incoming spamming might be concentrated to some domain or some email accounts. Most of the incoming spamming are normally caused due to the “catch-all” being set as main account. So, it is always better to set the “catch-all” parameter as fail.
WHM >>Main >> Server Configuration >> Tweak Settings:(Under “Mail” section)
This is the default catch-all/default address behavior for new accounts. “fail” is usually the best choice if you are getting mail attacks.
Also, enable RBL and spamassasin options in your cPanel server.(WHM >> Main >> Exim Configuration)
For outgoing spamming, users might be use PHP scripts to send spam mails. You can find the source of those PHP scripts using the command ‘ps -C exim -fH ewww |grep home‘.

Preventive measures to block spam include:
Limit the emails that can be sent.
Tweak the Mail server settings as Follows: WHM >>Main >> Server Configuration >> Tweak Settings:”The maximum each domain can send out per hour (0 is unlimited) : 300″
Some helpful exim commands include:
exim -bp(For showing the mail queue)
exim -bpc(Count of mail queue)
ps -C exim -fH ewww (For finding script sources )
exim -bp |exiqsumm (To get full exim queue summary )
exiqgrep -z -i | xargs exim -Mrm (Remove all frozen messages)

PHP Scripts Causing High Load
This can easily be found out through the results of the ‘pstree apu‘ and the ‘top -c‘ commands, during load spikes.

MySQL Abuse
You can track down MySQL usage using the command ‘mysqladmin proc stat‘.

DOS Attack (Denial Of Service)
By the name itself, we can understand what this would mean: denial of services like httpd, exim, ftp, etc. Mostly we deal with DOS attacks on Apache services, which may cause the sites to slow down or not load at all.
A DOS attack is a situation where there is a high number of simultaneous connections from a certain IP address or IP addresses. We can easily find the culprit IP by using the ‘netstat’ command. Once caught, it is better to block those IP’s using firewalls, without much delay.
The command for listing IP addresses and the number of connections is:
netstat -plan |grep :80|awk ‘{print $5}’ |cut -d: -f1 |sort |uniq -c |sort -n
The command for finding the total connections is:
netstat -plan |grep :80 |wc -l
If you do find any IP address having too many connections (more than 50), then you have a good reason to block it either by using iptables, APF, or CSF.
Using iptables, you can block an IP by issuing the command:
iptables -A INPUT -s –dport 80 -p tcp -j DROP
/etc/rc.d/init.d/iptables save
If you are using APF, this command may be used:
apf -d IP
Using CSF:
csf -d IP
DDOS is distributed denial of service, which is an advanced form of DOS attack. The number of connections from an individual IP might be low, but the connections will be from many IPs or a range of IPs.
Most of times attack will be concentrated to one domain. In such cases, its always prudent to kill the dns of domain to reduce the attack.
Preventive measures for DDOS attacks include:
Install firewalls like APF or CSF and configure them to block the high connection IPs.
Install mod_dosevasive module (only during the DOS attack)

Conclusion
As we all know, “prevention is better than cure”. So, it is always wiser to take necessary measures before our servers go out of control. Happy monitoring!!
Install System Integrity Monitor (SIM) or CSF and configure it to get alerts during load spikes.
Update your old and vulnerable software
Take all necessary preventive measures to avoid load spiking.
Secure your servers and tweak apache, exim, etc for better performance.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s