This server has been down more or less continuously for the last 24 hours. It was all part of the semi-annual celebration known as “Hack Day”, which is always accompanied by games such as “Whack a Hacker” and “Patch Roulette”.

This one was a bit more “creative” than usual. I’ve had several SQL injection defacements of parts of my site, but this time the little darlings managed to get some executable code onto my server. They then used that code to generate thousands of emails- thankfully the server crashed before too many of those went out.

I’ve got things back up with some figurative plywood over the holes the hackers made their entry through. I have a few more things to put in place before I’m satisfied that the server will last another year or so before someone wastes my time again. And if any whitehat security guy wants some out of date hacker scripts that probably first rode into the wild nearly two years ago, send me an email- I kept all the bits and bobs they left behind.

When I’m done, the server will be a bit harder to get into, which means that its starting to get more and more like a fortress with barbed wire, electric fences, and gun turrets- what a fun thing to have to deal with! But as the old saying goes, the only secure server is the one that’s unplugged.

UPDATE: I found the specific vector that the hackers used to get into my system, and the pattern of log file entries that shows the sequence the hackers went through to find the weakness. The thing to look for in the server HTTP log files is any URI containing the string “wget”. The weak spots that the hackers probed again and again were:

• awstats (an http stats program): that didn’t work out for them on my server since I password protect the CGI directory
• phpNuke and the phpBB forums software: this was my weak spot- I had an older version of phpBB that apparently had a vulnerability allowing changes to its configuration path. They then used this to download and execute a piece of PHP code. From there, they downloaded additional files that they stored in the /tmp directory.

From the looks of things, the phpbb forums vulnerability was used three times on my server- once on July 23rd by someone who installed the rOnin exploit package (a little bit of info can be found here. A second injection of the same code from a different site (this time in Brazil) took place on October 2nd. The third and final time, which is when the hackers began using my server in earnest to send email, took place on October 3rd, and I shut it down that night.