Determining how your site was hacked

The first step in getting things back to normal is to determine how the account was hacked. In general, almost all hacks occur through three methods ranked in order of likelihood:

  1. A CGI vulnerability in software you've uploaded to your website has been exploited and used to write/execute arbitrary code on the server.
  2. Your FTP/SSH password has been compromised.
  3. You have world-writable directory permissions on web-accessible folders.

Perhaps counter-intuitively, CGI hacks are more common than FTP/SSH password hacks primarily due to the sheer proliferation of pre-bundled software that people set up and frequently later forget to update. However, FTP/SSH hacks are the easiest to check up on. World-writable folders attacks are possible when the attacker is in the same machine that your account is. With world-writable permissions (o=w), anyone with an account on the hosting machine is able to put files there.

Looking for FTP/SSH hacks

Since recent FTP/SSH hacks are easiest to spot, start by eliminating this option. Log into your user via SSH and run the following commands.

Shows you your login history for the current month:

[server]$ last -i | grep youruser

Shows you your login history for the prior month:

[server]$ last -if /var/log/wtmp.1 | grep youruser

The usernames printed by "last" truncate after 8 characters, so if you have a longer username you'll want to truncate yours in the grep string as well.

The output looks something like this:

youruser pts/4        99.139.XXX.XXX   Wed May 28 06:10 - 07:11  (01:00)    
youruser pts/5        66.33.XXX.XXX    Sun May 25 09:31 - 12:14  (02:42)    
youruser ftpd30715    66.33.XXX.XXX    Wed May 21 14:16 - 14:16  (00:00)    
youruser pts/2        66.33.XXX.XXX    Tue May 20 13:22 - 14:18  (00:56)    
youruser pts/2        66.33.XXX.XXX    Tue May 20 13:06 - 13:22  (00:15)    

You can simplify this data to print out only IP addresses and counts by adding pipes to a few simple commands:

[server]$ last -if /var/log/wtmp.1 | grep youruser | awk '{print $3}' | sort | uniq -c

The output looks more like this:

4 66.33.XXX.XXX
1 99.139.XXX.XXX

You may find either method useful for determining who has logged into your FTP/SSH user.

The wtmp logs only go back at most 1–2 months, so if the hack is older than that, DreamHost won’t have any records of it.

If you've determined that FTP/SSH is the source of the hack, you should:

  1. Change your password. Visit the Change password article for further details.
  2. Discontinue use of FTP which sends your password over the Internet in plaintext, and then switch to SFTP or SSH. You can disable FTP for the account in the control panel on the same page that you changed your password.
  3. Ensure there is up-to-date virus/malware scanning on any computers on which you've used the password/user in question.

Looking for CGI hacks

Checking your logs on a specific date

If possible, start by trying to determine when the hack occurred. This can often be determined by noting the modification date of any files which have been modified by the hacker. Usually, they will all share a common modification time. If that time can be determined, check the logs for that day and time to see what was requested or posted to your site.

Careful examination of your log files often reveals exactly how your site was hacked. Every request and post to your site is logged and the log files can not be changed by the user, so the record of the attack is preserved provided that the exploit was discovered within the 30 days or so which are recorded in the logs.

Checking all of your logs

If the original time can not be ascertained, the log files can be scanned en masse although it's more difficult to uncover the evidence of the intrusion. You can run the following command on your web server after you log in via SSH:

[server]$ gunzip -c ~/logs/example.com/http/access.log.* | gawk '{a[$7]++}END{for (i in a) {print a[i]"\t"i}}' | sort -n | more

This returns a report of all requests to your site from all available logs sorted in ascending order. Since the majority of requests should (hopefully) be legitimate, it's easiest to start by looking at the top of the report — the least frequent requests — for evidence of the exploit:

It's not uncommon to see many requests which should not be there, as this is the nature of a website since it is publicly accessible. These requests do not, in and of themselves, indicate a breach, but should be investigated to see if the request successfully revealed a vulnerability.

Looking for world-writable directories

World-writable directories allow file writing by any user on the machine, and these directories can be mass-scanned. It's recommended that you perform this step anytime you suspect a breach.

Even if you're sure you didn't make any permission mistakes, some less security-aware software vendors or plugin developers often use system commands or language-native permission-management functions to make some directories (usually ones used for caching and temporary files, session files, and so on) to ease installation and management.

To scan for directories with world-writable permissions, use the UNIX find tool. Run this command via SSH in your domain’s web directory.

This command searches all folders below the folder in which you run it:

[server]$ find . -type d -perm -o=w

If no results are displayed, then no folders are world-writable.

Fixing world-writable directories

You can mass-change all your world-writable directories permission with the UNIX find tool. Run this command via SSH:

[server]$ find . -type d -perm -o=w -print -exec chmod 770 {} \;

When you run this command in your web directory, the following happens:

  • All directories that have 777 permissions are found.
  • Those directories with 777 permissions are changed to 770.
  • Other directories are not touched.

It's always better to enumerate all the world-writable directories and then deciding the proper permissions. Some folders require special attention.

See also

Did this article answer your questions?

Article last updated .