Using Cloudflare, an Amazon Load Balancer, with Nginx and Fail2Ban
I recently started using WordOps as the primary infrastructure solution for my WordPress sites. I’ve been quite happy with it as well.
I’ve been using Cloudflare for quite a while now, and have been extremely pleased with the product offering; even the free service is fantastic — I have also used Cloudflare as an Enterprise customer.
More recently though, I’ve started to dive into Amazon Web Services (AWS) and started using their Load Balancer technology in my high availability WordPress infrastructure configuration.
WordOps comes out of the box with a configuration file called cloudflare.conf inside of /etc/nginx/conf.d. This file comes pre-loaded with Cloudflare’s network of IPv4 and IPv6 addresses. Aside, WordOps regularly updates this list as well with Cloudflare’s public list of IP addresses.
What does this file of IP addresses to? It tells Nginx which IP addresses to recognize as a proxy so that it can listen for the real IP address behind the proxy.
Before the Fix
Instead of seeing a bunch of Cloudflare IP addresses in your access log files, you will instead see the real IP from visitors. This is much better for many reasons, but specifically one reason I’ll note below.
Fail2Ban is a great product which allows your system to more granularly control and protect against bad actors. I also have Cloudflare’s WAF in front, but doesn’t always protect against attacks like brute-forcing into wp-login.php.
I noticed that my website was suddenly, at random, showing Nginx “Bad Gateway 502” errors. I dug, and dug, and dug, and finally determined the cause.
While I had Cloudflare’s IP addresses “whitelisted” in cloudflare.conf, I did not have my load balancer IP addresses whitelisted. So instead of seeing the real IP addresses in my access logs, I was seeing the load balancer IP addreses. I didn’t think much of at it first, until I noticed that Fail2Ban was blocking my load balancer, which was in turn causing the bad gateway across my sites.
Solution
To resolve this discrepency, I added my AWS VPC subnet group, for example, 172.123.x.x, to my cloudflare.conf file. I tested the Nginx config — it passed — and then I reloaded Nginx.
Suddenly, I was seeing real IP addresses in my access log files and Fail2Ban could properly read who might be attacking to block the correct IP.
You may also be interested in the automated WordPress backup bash script I wrote, easily storing backups on Amazon S3.
I was wondering why WordOps had these bloated config files there. But your experience just reinforced my suspicion that it was all unnecessary . Because Cloudflare already has a “Real IP” option that passes on user IPs using headers, you just need to turn it on. Also, instead of installing more bloat like Fail2ban, you just can have basic rate-limiting in Nginx for sensitive URLs. And if you are running Ubuntu, their native UFW firewall is much easier anyways IMO for other flood type of IP address based attacks. I don’t understand why so many blogs still recommend Fail2ban.
See this SlickStack Nginx config:
https://github.com/littlebizzy/slickstack/blob/master/nginx/nginx-conf.txt
Some very good observations. Yes, rate limiting certainly helps.
Personally, I use Access by Cloudflare, as it completely limits access to the configuration I set. Whether it’s IP address/range, email address, @domain for e-mail, etc. And it completely blocks all access at the edge level, never even touching the origin server. I use this across many applications and it’s quite effective.
Fail2ban isn’t bloatware. heh.
You need fail2ban because it can interact with Cloudflare to add/remove the bans. If you just left everything up to nginx and ufw it would be all local. YES, cloudflare fwds you the real vistors IP but the actual requests are coming from Cloudflare IPs. So if you banned the user’s IP locally, it wouldn’t block them. You would need to block their IP on Cloudflare to stop them from accessing your services. If they were bypassing Coudflare, and had your systems IP address, then yes- ufw plays a roll.
Fail2ban is great with controlling policy- bans, unbans… and those controls can be any command, which is why people use it to manage their CLoudflare firewall via their API.