Secure Nginx and WordPress with Fail2ban

A few days after this self-hosted Nginx server and WordPress up and running, I started to see massive login attempts trying to gain access on SSH and WordPress login. After evaluating various solutions and plugins, I settled on using Fail2ban to block such attacks, and here are the steps on how I did it.

Fail2ban

Fail2ban is a daemon service based a simple and easy to understand concept. It involved three steps:

1. Monitoring log activities based on a pattern pre-defined with a regular expression (i.e. a filter);
2. Check against ban criteria such as number of attempts (i.e. jail rule) for a matched pattern;
3. Create an iptables rule to block the related ip for a period of time

So to setup an effective fail2ban defence is about creating a filter using regular expression to monitor a log activities; For which log, which port or protocol, how often, and how many times when the monitored pattern occured before blocking the IP, will determined by setting up the “jail” rule.

Installation of fail2ban is very simple and straightforward with a command line:

sudo apt-get install fail2ban

Fail2ban starts by itself as a daemon upon installation. Fail2ban configuration file can be found at /etc/fail2ban/fail2ban.conf. Fail2ban also come with many pre-configured filters in the /etc/fail2ban/filter.d directory. The jail rules are stored in a single file called jail.conf in /etc/fail2ban/ directory. It is quite a common practice to keep the original jail.conf file as is, and duplicate the file and rename it as jail.local and only make changes on jail.local.

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

There are several preset default settings under the [DEFAULT] section of the /etc/fail2ban/jail.local (which we just copied from jail.conf), all those default or global settings can be overrides under each jail setup. So let’s first take a look at those default settings that you may want to adjust in according to your requirements.

[DEFAULT]

ignoreip = 127.0.0.1/8
bantime  = 600
findtime = 600
maxretry = 3

banaction = iptables-multiport
action = %(action_)s

The ignoreip allows you to setup the IPs that fail2ban should ignore, for example, your local IPs or your home or office public IP if that’s where you are going to access the server. You can simply add the IP and range separate with a space in the line:

ignoreip = 127.0.0.1/8 192.168.0.1/24

The bantime determined how long in second where you’d want to block the IP, the default is 600 seconds or 10 minutes.

The default value for findtime is 600 and 3 for maxretry, meaning that a host would be based if it failed for maximum of 3 retry within 10 minutes (600 seconds) period. It determined how often and how many times you’d allows a user to access the host.

The default action parameter banaction is iptables-multiport. The action defines the string that is going to pass to iptables for setting up the iptables entry. Fail2ban provides three presented action shortcuts, but by default, the ban-only action (action_) is used.

# The simplest action to take: ban only
action_ = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]

SSH Access

Ideally it is much safer to keep the SSH access for internal IP only, but sometime it is necessary to keep the SSH port accessible from public IP so that I can access it remotely. SSH generally attracted a lot of hacking trying to guess the username and password.

By default, fail2ban have the SSH Jail section enabled upon running with the setting shown in the /etc/fail2ban/jail.local as:

[ssh]

enabled  = true
port     = ssh
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 6

The jail monitoring /var/log/auth.log entries on ssh port based on the filter set in /et/fail2ban/filter.d/sshd.conf and will take action if the remove access failed to provide the correct credential for 6 times (which by the way, override the global default setting of maxretry = 3, and you might want to adjust it).

The _daemon = sshd in /etc/fail2ban/filter.d/sshd.conf means fail2ban is running as a daemon. The failregex is a collection of regular expression for various auth log scenario to determine the illegal or unsuccessful login attempts.

[Definition]

_daemon = sshd

failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|error) for .* from ( via \S+)?\s*$
            ^%(__prefix_line)s(?:error: PAM: )?User not known to the underlying authentication module for .* from \s*$
            ^%(__prefix_line)sFailed \S+ for .*? from (?: port \d*)?(?: ssh\d*)?(: (ruser .*|(\S+ ID \S+ \(serial \d+\) CA )?\S+ %(__md5hex)s(, client user ".*", client host ".*")?))?\s*$
            ^%(__prefix_line)sROOT LOGIN REFUSED.* FROM \s*$
            ^%(__prefix_line)s[iI](?:llegal|nvalid) user .* from \s*$
            ^%(__prefix_line)sUser .+ from  not allowed because not listed in AllowUsers\s*$
            ^%(__prefix_line)sUser .+ from  not allowed because listed in DenyUsers\s*$
            ^%(__prefix_line)sUser .+ from  not allowed because not in any group\s*$
            ^%(__prefix_line)srefused connect from \S+ \(\)\s*$
            ^%(__prefix_line)sReceived disconnect from : 3: \S+: Auth fail$
            ^%(__prefix_line)sUser .+ from  not allowed because a group is listed in DenyGroups\s*$
            ^%(__prefix_line)sUser .+ from  not allowed because none of user's groups are listed in AllowGroups\s*$

The SSH fail2ban filter is quite comprehensive and works without changes. The only thing that need to be customised is the maxretry, bantime and findtime. In addition to setting up fail2ban, You should tighten your server security using SSH access key, disables password entry and root user access, you could then set a very short findtime window, with max retry of 1 and very long bantime to block most of the login attacks.

WordPress Login

WordPress is extremely popular, brute force attacks against WordPress have therefore always been very common. These login attempts often come from botnets, they are automated and their goal is break into as many websites as possible by guessing their passwords. Those attacks generated hundreds or thousands of login attempts per day.

Before using fail2ban to protect the host from brute force attacks, there are a couple of changes to make on wp-login page. When key in username and password at work-login page, WordPress will generates error message indicating whether you have an invalid username or password, this provides a good user experience to general users, however it is too "friendly" for hackers. We'd want to have a more generic error message simply as "invalid credential" when a login attempts failed. To change that we can added the following PHP function either in functions.php (if you are using a child theme) or as a MU-plugin, in my case, I simply added into my customise plugin that I created.

/* Change WP-Admin login error message to be more generic */
function generic_login_msg ($msg) {
    global $errors;
    $err_codes = $errors->get_error_codes();
    if ( in_array( 'invalid_username', $err_codes ) || in_array('incorrect_password', $err_codes )) {
        $msg = 'ERROR: Invalid credential.';
    }
    return $msg;
}
add_filter('login_errors', 'generic_login_msg');

Another issue about wp-login is that when an incorrect username or password is entered, the page still return a 200 http status code which is the same status used for a successful login authentication. Konstantin Kovshenin suggested to generated a 403 status code to failed login attempts.

/* Return 403 instead of 200 when wp-login failed */
add_action( 'wp_login_failed', function () { 
    status_header(403); 
} );

Fail2ban does not have a pre-set filter for WordPress, so we will have to create ourself. Create a filter name wp-login.conf in /etc/fail2ban/filter.d directory:

#Fail2Ban filter for wp-login failures

[Definition]

_daemon = wp-login
failregex = <HOST>.*POST.*(wp-login\.php|xmlrpc\.php).* 403 

# based on https://konstantin.blog/2014/fail2ban-wordpress-nginx/

The failregex monitors nginx access log and matches a POST request to wp-login.php or xmlrpc.php with a status code of 403. Open /etc/fail2ban/jail.local using editor and add the following jail rule at the end of file now.

[wordpress]

enabled  = true
port     = http,https
filter   = wp-login
logpath  = /var/log/nginx/access.log
maxretry = 3
bantime = 7200

Our wp-login filter would monitor the nginx access log file for the login attempts that generated 401 error status, and when it happened for more than 3 times, it will ban the access of that pareticlar ip for up to 2 hours (7200 seconds).

Restart the fail2ban daemon for the new settings to take effect:

sudo service fail2ban restart

Please noted that each time the fail2ban is restarted or when the host is reboot, all the bans are released and restart again.

Check both fail2ban.log and daemon.log to make sure that there is no error generated by the newly created configuration. You can also check iptables to make sure everything is setup correctly.

sudo iptables -L

If everything setup correctly, you should see the following iptables entries:

fail2ban-wp-login tcp -- anywhere anywhere multiport dports http,https
fail2ban-ssh tcp -- anywhere anywhere multiport	dports	ssh

Nginx IP blacklist

As fail2ban only block the ip with failed login attempts for a period of time based on bantime settings. For those regular offenders, I’d like to block it permanently. To do that, I use awk utility to find out who are the regular offenders, and how many attempts occured.

cd /var/log/nginx
awk -F\" '($2 ~ "/wp-login.php"){print $1}' access.log | awk '{print $1}' | sort | uniq -c | sort -n

This will generated a list of IP addresses prefix with a number which indicated the number of attempts trying to access wp-login.php. I then created a blacklist of ip address configuration file /etc/nginx/blockips.conf, add those regular offenders into the deny list:

deny xxx.xxx.xxx.xxx;
deny yyy.yyy.yyy.yyy;
deny zzz.zzz.zzz.zzz;
…

Then add the list under the http directive in /etc/nginx/nginx.conf file:

http {

    # Block blacklisted IPs
    include /etc/nginx/blockips.conf;

    #rest of the configuration directives
    . . . . .

}

Verify and reload the nginx configuration

sudo nginx -t
sudo service nginx reload

That's all for a simple but effective solution for stopping brute force attacks on Nginx and WrodPress.