Tuesday’s Technical Problems

On Tuesday, you may have noticed that the Huletts Current went down. This was caused by a widespread attack on WordPress login pages. The attack was a large one (hundreds of hits a second to many WordPress sites spread across the Internet). At that point, the fastest solution was to drop all traffic to the login page while the hosting company looked for solutions.

The downside to this, of course, is that it blocked legitimate access to me when I wanted to login on Tuesday.

The hosting company acted quickly to put new security rules in place that would help them allow legitimate customers to login to WordPress, but would stop the attacker after a number of bad attempts. During some periods of the week the site itself went down.

These changes were rolled out Tuesday afternoon. It took a few tries to find the right balance to block the bad guy but not keep a legitimate users from logging in. The attack subsided overnight.

The attack returned in force on Wednesday during the business day. This made it obvious that the attack was based off a botnet—likely using the computers of unsuspecting office workers coming in for a normal day of work!

By this point, the hosting company and their providers were flagging almost one hundred thousand IP addresses, and more new IP addresses were showing up every second. Even though much of the attack was being thwarted, it was so large that simply handling the traffic was starting to impact the hosting servers.

The team was able to keep things stable for most of Wednesday, working hard to tweak rules as they identified new trends.

By Thursday, it was clear that the attack was not subsiding. However the breakthrough happened on Thursday, as a difference was found between the way the attack accesses WordPress and legitimate customers access WordPress. That change was rolled out on Thursday afternoon and hundreds of hits a second dropped to nearly none.

We head into the weekend in good shape, but vigilant against a returning or altered attack. In the meantime, if you are feeling any lingering effects (the most common one might be if your IP got marked as a possibly bad IP) please let me know.