Securing Your Server using IPSet and Dynamic Blocklists

As a dedicated server or virtual private server (VPS) owner, one important task is to defend against online attacks. IPTables allows a sysadmin to filter traffic by configuring the tables in the Linux kernel firewall. In this tutorial, I will discuss how to use IP sets with dynamic blocklists to better secure your server.

IPSet Logo

IP sets are a framework inside the Linux kernel, managed by the IPSet utility. It can be used on most servers except OpenVZ VPS. Compared to "vanilla" IPTables, using IP sets could improve efficiency when filtering hundreds of thousands of IPs. Unlike normal IPTables chains which are stored and traversed linearly, IP sets use indexed data structures to make lookups very efficient, even for large sets.

The methods discussed below should be "complementary" to your existing IPTables rules, but not to replace them. To learn more about how to create basic IPTables rules, search for related tutorials on the web, or see my guide on securing the Asterisk VoIP server.

As a side note, check out my tutorial on setting up your own IPsec VPN server with both IPsec/L2TP and Cisco IPsec.

To install IPSet (and Wget, will need later), use these commands.

For Ubuntu/Debian:

apt-get update  
apt-get install ipset wget  

For CentOS/RHEL:

# Enable the EPEL repository  
yum install epel-release  
yum install ipset wget  

How can we better secure servers with IPSet? The answer is to take advantage of dynamic blocklists maintained by various security research companies or groups on the Internet, and deny access to these "bad" IPs or IP networks. Here is an example collection of such lists:

Blocklists of Suspected Malicious IPs and URLs

Now, how to actually use these blocklists with IPSet, and update them at set intervals? I came across a very useful script on the Gentoo Forums, created by author "Bones McCracker". Here's a link to his original post, with some good explanations on IPSet. For your convenience, his first script is shown here, which works with the DShield Recommended Block List:

To save the script, click on "view raw" at bottom-right corner, Ctrl-A to select all, Ctrl-C to copy, then paste into your favorite editor. Alternatively, you can download it. In order to enable filtering using the IP set produced by the script, an IPTables rule must be added. One way to do this is to simply append these lines to the end of the script, and save it:

iptables -nL INPUT | grep "block src" &>/dev/null  
if [[ $? -ne 0 ]]; then  
  iptables -I INPUT -m set --match-set block src -j DROP
  iptables -I INPUT -s YOUR_IP_ADDRESS -j ACCEPT
fi  

(Update: Thanks to this Reddit user for suggesting to whitelist one's own IP in case your Internet Service Provider (ISP) appears on a list. I have added a rule to address this. Replace YOUR_IP_ADDRESS above with your public IP.)

Also, for added security you can change the target line to use "https":
(Note: May not work with some Wget versions! Test it before using.)

target="https://www.dshield.org/block.txt"  

After making the changes, save the script as e.g. /root/ipset_dshield.sh, mark it executable, and then test it. If everything works as expected, proceed to add a line to your /etc/rc.local so that the script runs automatically upon reboot. The last step is to create a cron job to fetch the list at set intervals and update your IP set. For example:

# /etc/cron.d/update_block  
# Global variables
SHELL=/bin/bash  
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin  
MAILTO=root  
HOME=/  
# Every 15 minutes, poll for update to dshield
# block list, and update firewall blacklist.
13,28,43,58 * * * * root /root/ipset_dshield.sh &>/dev/null  

To check the contents of the IP set we just created, run the following command. Note that this set was named block:

ipset list block  

Now, we move on to another useful script created by the same author, which works with the IPv4 Fullbogons List by Team Cymru. To help explain what a Fullbogon is, here's a quote from their project page:

"A bogon prefix is a route that should never appear in the Internet routing table. A packet routed over the public Internet (not including over VPNs or other tunnels) should never have a source address in a bogon range. These are commonly found as the source addresses of DDoS attacks.

Bogons are defined as Martians (private and reserved addresses defined by RFC 1918, RFC 5735, and RFC 6598) and netblocks that have not been allocated to a regional internet registry (RIR) by the Internet Assigned Numbers Authority. Fullbogons are a larger set which also includes IP space that has been allocated to an RIR, but not assigned by that RIR to an actual ISP or other end-user ...

It is important to realize that the bogon and fullbogon lists are NOT static lists ... make sure that you have a plan for keeping your filters up-to-date, or within a short space of time you will be filtering legitimate traffic and creating work for network administrators everywhere. This is especially true for the fullbogons list, which has significant changes every day."

Again, for your convenience here is the script:

To save the script, click on "view raw" at bottom-right corner, Ctrl-A to select all, Ctrl-C to copy, then paste into your favorite editor. Alternatively, you can download it. Append the lines below to the end of the script to add the required IPTables rules. Note that I put three extra rules with RETURN to exclude certain private IP networks. Please carefully review them and remove those not in use on your server:

iptables -nL INPUT | grep "IP4BOGONS" &>/dev/null  
if [[ $? -ne 0 ]]; then  
  iptables -N IP4BOGONS; iptables -F IP4BOGONS
  iptables -A IP4BOGONS -s 10.0.0.0/8 -j RETURN
  iptables -A IP4BOGONS -s 172.16.0.0/12 -j RETURN
  iptables -A IP4BOGONS -s 192.168.0.0/16 -j RETURN
  iptables -A IP4BOGONS -m set --match-set fullbogons-ipv4 src -j DROP
  iptables -I INPUT ! -i lo -j IP4BOGONS
fi  

As mentioned in the quote above, it is essential that you keep the Fullbogons list up-to-date. Here's an example cron job:

# /etc/cron.d/update_bogons  
# Global variables
SHELL=/bin/bash  
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin  
MAILTO=root  
HOME=/  
# Every four hours, poll for update to ipv4-fullbogons
# block list, and update firewall blacklist.
55 3,7,11,15,19,23  * * * root /root/ipset_bogons.sh &>/dev/null  

So far I have presented two IPSet update scripts, which can be used with the DShield Block List and Team Cymru Fullbogons List, respectively.

Another security research company maintains the ETOpen Ruleset, which is also free (as in beer). View the current rules. To use it with IPSet, see example update scripts. I currently use the second one by Thomas Mueller, but had to fix a few bugs to make it work.

Follow this link to view my modified version of the update script:
(Replace YOUR_IP_ADDRESS at the end with your actual public IP)
https://gist.github.com/hwdsl2/7977b49f7257fe516b60

And here is an example cron job to update this ruleset:

# /etc/cron.d/update_em  
# Global variables
SHELL=/bin/bash  
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin  
MAILTO=root  
HOME=/  
# Every four hours, poll for update to the emerging threats
# ETOpen block list, and update firewall blacklist.
55 3,7,11,15,19,23 * * * root /root/ipset_em.sh &>/dev/null  

To learn more about the detailed usage of IPSet, read the manual page or search for "IPSet tutorial" on Google.

Finally, you can list the IPTables rules we added with these commands. Check the relevant counters after some time and they should increase, which indicates successfully blocking of "bad" traffic using the lists:

iptables -nv -L INPUT; iptables -nv -L IP4BOGONS  

In addition to those discussed above, there are other useful blocklists such as this one by StopForumSpam (example IPSet script). While I have not personally used this or any of the other lists, you can certainly try to adapt the update scripts mentioned in this article to work with them.

This concludes my discussion on using IPSet and dynamic blocklists to protect your server. I hope you will find it useful!

Please share this post if you like it, and do not hesitate to write your comments or questions in the Disqus form below.


Next article: Get Two Public IPs on an Amazon EC2 Instance for Free
Previous article: IPTables GeoIP, Port Knocking and Port Scan Detection

Return to Lin's Tech Blog Homepage

Lin Song

Fresh PhD graduate in Electrical and Computer Engineering (ECE). As a hobby, I love computers, Linux and programming.  LinkedIn Profile  GitHub



View or Post


Disclaimer: All content provided on this blog is for informational purposes only. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. All trademarks mentioned herein belong to their respective owners.
    The owner of this blog will not be liable for any errors or omissions in this information nor for the availability of it. The owner will not be liable for any losses, injuries, or damages from the display or use of this information.

Your name:

Email address:

Website URL:

Please leave a comment:

You agree that this form is for A N T I-S P A M B O T S!
     D O-N O T-S U B M I T !