How do we know that the server is under DDOS attack?

DDOS attack measures

How do we know that the server is under DDOS attack?

We can confirm it by checking the result of netstat command:

netstat -an|awk ‘/tcp/ {print $6}’|sort|uniq -c

This will show the states and number of connections at that time. The different states that are visible mostly in servers are:

1. ESTABLISHED – This will be legitimate connections established to the server
2. SYN_SENT – The client will be actively attempting to establish a connection.
3. SYN_RECV – A connection request has been received from the network.
4. FIN_WAIT – The socket is closed, and the connection is shutting down.
5. TIME_WAIT – The socket is waiting after close to handle packets still in the network.
6. LISTEN – The socket is listening for incoming connections.
7. LAST_ACK – The remote end has shut down, and the socket is closed. Waiting for acknowledgement.

If the number of connections in SYN_SENT, SYN_RECV, TIME_WAIT, FIN_WAIT are very large in the rate of 1000s then the server is surely under attack.

As a first step we can tweak the values set for SYN_SENT, SYN_RECV, TIME_WAIT, FIN_WAIT in the file /etc/sysctl.conf. Reduce the value of net.ipv4.tcp_fin_timeout to 3 or 5. Normally it will be set to 120 as default. Make the following changes in /etc/sysctl.conf

# Enable TCP SYN cookie protection
net.ipv4.tcp_syncookies = 1

# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 3

# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0

# Turn off the tcp_sack
net.ipv4.tcp_sack = 0

Then execute the command :

sysctl -p

Then we will have to find out how the attack is being performed, is it from any particular IP or from large number of IP addresses to the server. If it is from any particular IP to the server, then we can fix it by blocking the IP in the firewall. If it is from a large number of IP with one or 2 connections then we will have to find more details to stop it. But will will not be able to completely stop the DDOS attack, we will have to tweak some settings in the server so that the number of connections can be reduced.

Once we reach the result that the server is under attack by checking the number of connections in different state, we need to find to which port the attack is being done. Suppose the number of connections in state SYN_RECV is large. Then we can get the details using the following command:

netstat -lpan | grep SYN_RECV | awk ‘{print $4}’ | cut -d: -f2 | sort | uniq -c | sort -nk 1

The result will be the number of connections and the port open in the server. If the second field is 80 then the attack is to apache port.

In addition to the netstat command, you can use tcpdump command to find out if there is dos attack to a particular port.

tcpdump -nn -tttt -i any port 80

Similarly you can give different ports to find out to which port attack is being done. For example, port 53, 25 etc.

Once you understand the port you need to figure out is the attack done on a particular domain or IP. Suppose the attack is done on port 80, then we can tweak the apache settings as follows:

1. Increase the MaxClients so that we can prevent the condition of apache reaching its limit, since apache could not serve new requests. MaxClients can be set to a max value of the limit set in ServerLimit
2. Set KeepAlive on to set the KeepAliveTimeout
3. KeepAliveTimeout value to be reduced to 3 or 5

So the settings will be as follows:

MaxClients 500
KeepAlive On
KeepAliveTimeout 3

/etc/init.d/httpd restart

In order to narrow down the issue, we need to find out if the attack is on any particular IP in the server. This can be found using the following command:

netstat -lpan | grep SYN_RECV | awk ‘{print $4}’ | cut -d: -f1 | sort | uniq -c | sort -nk 1

After confirming the attack to the IP, we need to find out if the attack is made to a particular domain in that IP or to the IP as a whole. For that, you can check the apache error logs or top command. If in the apache error logs, you are finding the errors for a particular domain, then you will have to perform steps to prevent attack to the domain. For that we can perform the following steps:

1. We can block the connections to the domain using modsecurity. CSF is connected to modsecurity so that if we write rule to block a domain, the IP from which connections to the domain are made will be blocked. Since it is DDOS attack, there will be many IPs connecting to the server and blocking high number of IP addresses can cause load in the server and thus server can go down. In order to prevent that, you will have to first block the checking of modsecurity in lfd.

In /etc/csf/csf.conf, set the following:

= “0”

csf -r

Then, in the modsecurity configuration file, you can add the following:

SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer “domain\.com”

2. You can block the acesses to port 80 of the domain in the firewall using the following command:

iptables -I INPUT -p tcp –dport 80 -m string –string “domain.com” –algo bm -j DROP

3. If the connections are still not getting reduced, then you can limit the number of connections to the domain using bandwidth module as follows:

/scripts/setbwlimit –domain=domain.com –limit=256000

4. If nothing helped, you can nullroute the IP using the command:

iptables -I INPUT -d XX.XX.XX.XX -p tcp –dport port -j DROP

If the domain is having dedicated IP, then there is no need of above steps, you can directly make the IP down, by deleting the IP from the /etc/ips and restarting ipaliases. But in case of main shared IP, this cannot be done. We will have to reduce the TTL of the domains and change all the domains except the domain to which attack is being made to a free IP after 4 hours and then make the IP down after that so that the attack will be there for only 4 hours. But in such cases there will be issue with cpanel license etc. We will also have to make sure of the name server setting of the domain to which attack is being made. If the domain is using remote name servers, then we cant change any DNS setting of the domain in the server.

In order to prevent this in future, you can add the following commands:

iptables -A INPUT -p tcp –tcp-flags SYN,FIN SYN,FIN -j DROP
iptables -A INPUT -p tcp –tcp-flags SYN,RST SYN,RST -j DROP
iptables -A INPUT -p tcp –tcp-flags FIN,RST FIN,RST -j DROP
iptables -A INPUT -p tcp –tcp-flags ACK,FIN FIN -j DROP
=-=-=-=-=-=-=-

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s