Although all WordPress users should upgrade right away, we’ve added security rules to our servers to protect our Web hosting customers who haven’t yet upgraded. Other people may find the rules useful if they use mod_security on Apache Web servers. The rest of this post contains more technical details.

The WordPress bug

The exact nature of the security flaw is described in this blog post. Simply put, if an author of a post creates a trackback that contains a single quote character in WordPress 3.0.1 or earlier, they can make WordPress run database commands of their choosing. Those database commands could be used to make the author an administrator, for example.

To block this, we’ve added a mod_security rule that prevents anyone from creating a trackback URL that includes a single quote character (which is extremely unlikely to occur legitimately):

SecRule SCRIPT_BASENAME "^post\.php$" "deny,status:412,auditlog,chain"
SecRule ARGS_POST:trackback_url "'"

Again, we’ve already applied this rule to our servers, so our Web hosting customers are protected. But if you run a server that hosts WordPress blogs, you should consider adding this mod_security rule to protect your users, too.

There are many sources of missing image file references. Themes and plugins can generate them. Your site may be missing a favicon.ico file. Some RSS news readers make a lot of requests for files named apple-touch-icon.png and apple-touch-icon-precomposed.png, which most Web sites do not have. Check your Web site’s analytics report (such as AWStats) or raw log files to see what image files are missing from your site.

If you’re a Tiger Technologies customer running WordPress, you already benefit from a global rule we implemented for missing favicon.ico files (as discussed in a previous blog post). We’ve also recently updated our WordPress Performance page with extra tips on returning faster “404″ results for other missing files such as robots.txt and for all missing images.

Here’s another tip that can help dramatically: Remove “bot”, “ia_archive”, “slurp”, “crawl”, “spider” and “Yandex” from the Rejected User Agents box in the WP Super Cache plugin settings. (In most cases, this will leave the box completely empty.)

Those “Rejected User Agents” prevent cached pages from being created when a search engine “visits” your site. The FAQ says this is because there’s no point creating a cached file for a page that isn’t popular, which may be true for sites that don’t have many posts and aren’t that busy. The author of the plugin is wisely being conservative to avoid problems on hosting companies that use “NFS” disks where saving a cached file can be slow.

We don’t use NFS like that, though. And on a busy site with lots of archived posts, there’s a very good chance that a page that’s indexed by a search engine will be reindexed or viewed several times within the next couple of days. Allowing these pages to be cached can make a big difference. Besides, there’s no good reason not to cache a copy of the page, since you should be sending the same page to search engines as you do to actual users.

We removed those “Rejected User Agents” on a busy site with several thousand archived posts, with the “Expire time” set to 48 hours (172800 seconds) as we recommend, and here’s what happened to that site’s CPU usage:

As you can see, the average load dropped by almost half as more and more pages were cached. That’s another way of saying that the average page loading speed almost doubled. Those are pretty good results for a change that took just a few seconds to make.

By default, each WordPress blog is configured to send the login username and password as plain (unencrypted) text. If a hacker can see what you are sending during your login, they can easily steal your username and password. This can happen if you have a virus installed on your computer. It can also happen if your computer is virus-free but connects via WiFi. If your main computer uses a wireless connection, or if you or other users of your blog ever login with their laptops — blogging from a coffee shop, anyone? — remember that these connections can be insecure, and could be susceptible to revealing your password.

You can protect your blog by installing an “SSL certificate” and configuring WordPress to require secure logins. Your browser will then encrypt your username and password so that no one can intercept them.

Traditionally, only online stores used SSL certificates because they were very expensive. But SSL certificate prices have dropped quite a bit recently, and they’re now low enough that we think SSL certificates should be widely used to protect all logins and other sensitive data.

If you are a Tiger Technologies customer, you can get an SSL certificate for a great price. (One type of certificate, a “self-signed certificate”, is even free if you’re already on our Gold or Platinum hosting plans.) If you’re not a Tiger Technologies customer, you can search for companies selling SSL certificates or search for free self-signed certificates.

Configuring WordPress

Once you have an SSL certificate installed on your site, it’s easy to configure WordPress to use secure logins. Simply add this line anywhere to your wp-config.php file (after the opening “<?php” line):

define('FORCE_SSL_ADMIN', true);

This will ensure that your username and password are submitted to WordPress securely; all of your subsequent work (creating posts, etc) will be secure as well. You’ll see your Web browser’s “padlock” icon when you are using a secure connection. The WordPress “Administration Over SSL” page has more details.

Several shared hosting companies apparently allow customers to view the text of other customer’s files by default, and that allows malicious customers to discover the database password of another site (from the “wp-config.php” file) and alter the site.

Some security researchers think this is a WordPress flaw, but we agree with the WordPress folks that that’s nonsense. The real problem is that these hosting companies are allowing customers to view each other’s files, which shows a reckless disregard for security.

For the record, this kind of attack is impossible on our servers. We prevent customers from viewing each other’s files. In particular, we use suEXEC to run each user’s scripts under a separate Unix user ID, and we ensure that your files are protected from other users at the file system level.

In fact, we go a step further than even most reasonably secure companies: we protect your files even if you change every possible file and directory to be world-writable (mode 777), using an additional layer of Linux “access control list” rules on your top-level directories. We also automatically reset the bad directory permissions for you, just to be sure there are always two levels of protection.

So you don’t need to worry about this kind of attack; “we’ve got your back”.

A different attack on wp-config.php

Separately and coincidentally, our security systems detected a different type of attack on a “wp-config.php” file today. Some “hackers” on the Internet are trying to load files named “wp-config.php~” with a tilde character at the end. In most cases, that file won’t exist, but if you’ve edited your “wp-config.php” file with some text editors, it creates a “wp-config.php~” version as a backup copy.

This is an interesting attack. Of course, it’s not possible to load this file in a Web browser and see the source:

http://example.com/wp-config.php

However, it is possible to see the source of this file if it exists:

http://example.com/wp-config.php~

That’s because the server doesn’t recognize that the “.php~” file contains PHP code, so it shows the visitor the contents (source) of the file, with disastrous results because it contains passwords.

We’ve fixed this potential attack by blocking all requests for filenames ending in “.php~” on our servers, since it appears that nobody is using this legitimately.

If you run your own Web servers, this simple mod_security rule will do the trick:

SecRule REQUEST_FILENAME "\.php~$" "msg:'PHP file backup exploit',deny,status:412,auditlog"

(Just so it’s clear, we’ve already done this for our customers; that rule is mentioned in the hope that it might be useful to someone else. We also mentioned this issue on the WordPress support forums.)

But recently we’ve seen a number of our customers getting hammered by a ton of requests from FeedBurner. Usually the request is of this form:

/somepost?utm_source=feedburner&utm_medium=feed&utm_campaign=SomeCampaignString

We’ve also seen FeedBurner going crazy and making thousands of duplicate requests. One of the sites we host has gotten 45,000 simple status requests (HTTP “HEAD” requests) from FeedBurner today, for no good reason that we can see.

Unfortunately, the default rules of WP Super Cache prevent it from caching any request with a query that contains an equal sign. So all of these requests are being unnecessarily run freshly each time, rather than being served from the cache.

There’s an easy fix to this. Open your Web site’s .htaccess file. Look for the section of lines for WP Super Cache, and find the line which tests %{QUERY_STRING}. Insert this new line of text immediately above the existing line:

RewriteCond %{QUERY_STRING} ^utm_source=(feedburner|twitterfeed) [OR]

The new line (ending with [OR]) must come before the existing %{QUERY_STRING} line. After inserting, the two lines should look exactly like this:

RewriteCond %{QUERY_STRING} ^utm_source=(feedburner|twitterfeed) [OR]
RewriteCond %{QUERY_STRING} !.*=.*

There are four very similar blocks right next to each other in the .htaccess file. Be sure to add the new line to the same place in each block.

That’s the only change you’ll have to make! WP Super Cache will now be able to cache requests for normal pages that come from FeedBurner or Twitterfeed. If your Web site was being abused by FeedBurner (or Twitterfeed), you should see a definite improvement.

Update April 2012: Newer versions of the WP Super Cache plugin have four places that need to be changed, rather than two places. We’ve updated the post to reflect this.

In some cases, you might be storing data that’s not quite so important. And if it means your application can run much faster, you might be willing to risk a very small chance of data loss. That’s where MySQL’s “INSERT DELAYED” statement, which works with MyISAM table types (but not InnoDB tables), can be useful. (Tables are created as type MyISAM by default, so most tables are eligible to benefit from this tip.)

Adding the word “DELAYED” to your statement tells MySQL to remember the data to be added and return immediately to your application. MySQL will then write the data as soon as the database isn’t busy. This lets your insertion happen (effectively) immediately, and reduces the load on the database (and server). Using this technique can give your application a huge performance gain.

Since the record is not written immediately, there is a very small chance that the data will be lost before it’s written to the disk. However, the odds of this happening are very small. It would only happen if MySQL crashed before it had a moment of idle time to write out the record, if the server lost power, or if some similar unexpected event happened.

Using “INSERT DELAYED” is recommended for applications that do not absolutely depend upon records being immediately written. Any data which is only referred to at a later time (rather than on-demand) or in a summary fashion is a good candidate for using “INSERT DELAYED”. Some examples are logging the IP address of each visitor, or tracking ad impressions for each page viewed on your Web site. Be sure to read the official documentation for full details and additional considerations.

If you want to use “INSERT DELAYED”, first check the documentation to see if your application already supports it. If it does not, you’d need to modify the application. If you have any questions about whether a particular database table is a good candidate for using “INSERT DELAYED”, just ask us and we’ll take a look.

That advice makes sense if you have a sudden surge of traffic to a single page. However, if your site is generally very busy across all pages (for example, if you have an archive of hundreds or thousands of posts that are constantly being indexed by search engines), we’ve found that you should do the opposite to improve performance: set it much higher. We recommend setting it to 172800 seconds (which is 48 hours). This can cut your CPU usage in half, which will speed up your site.

The reason for this is that when WP Super Cache creates a cached page, it wants to make sure that those pages don’t build up forever. Every ten minutes or so, it looks through them all and deletes any that are older than the “expire time”.

On some servers that use a network file system called “NFS”, looking through a large number of files causes performance problems. That’s why the WP Super Cache author recommends making them expire quickly: it reduces the number of files it has to examine each time.

On our servers, we don’t use NFS and looking through lots of files does not cause a performance problem. Leaving the files for a longer time is safe and increases the chance that a page will already be cached when it’s needed.

If you’re a Tiger Technologies customer who makes this change and you want to see how it affects the CPU usage, just let us know and we can provide you with details.

Although all WordPress users should upgrade, we’ve added security rules to our servers to protect our Web hosting customers who haven’t yet upgraded. Other people may find the rules useful if they use mod_security on Apache Web servers. The rest of this post contains more technical details.

Surprising Apache script behavior

One of the fixes in WordPress 2.8.6 compensates for a peculiar and surprising feature (or bug, if you prefer) in the Apache Web server: in many cases, it will run a file with a name like “image.php.jpeg” as a PHP script.

Some software (like WordPress 2.8.5 and earlier) can be configured to allow strangers to upload JPEG and other files to your site — but it checks only the “.jpeg” file extension to see if the uploaded file is safe. So a “hacker” could upload a malicious PHP script named “image.php.jpeg”, then “view the image” in a Web browser… but the server would actually run the PHP script.

Because of that, we’ve added a mod_security rule that prevents site visitors from requesting certain file extensions that include “.php.” in their name. (There are other possible solutions, but testing has shown this is the least intrusive to our existing customers.)

Here’s a mod_security rule that accomplishes that (adjust the extensions to suit your taste; these are the WordPress 2.8.5 allowed extensions):

SecRule REQUEST_FILENAME "\.php[456]?\.(asf|asx|avi|bmp|gif|ico|jpe|jpeg|jpg|png|tif|tiff|wax|wmv|wmx)$" "deny,status:412,auditlog"

WordPress brute force attacks

Another recently reported attack against WordPress (which is unrelated to version 2.8.6 in particular) involves “brute force” attempts to guess the administrator password. The proper solution is to choose a good password for your blog (not a word from the dictionary!), but some people don’t do that.

To reduce the risk of successful attacks against our customers, we limit each IP address to 25 “wp-login.php” attempts within a five minute period. Here’s how you can do that with mod_security:

SecAction phase:1,initcol:ip=%{REMOTE_ADDR},nolog
SecRule REQUEST_LINE "post .*/wp-login" "nolog,phase:1,setvar:ip.wordpress_login=+1,deprecatevar:ip.wordpress_login=5/60"
SecRule IP:WORDPRESS_LOGIN "@gt 25" "deny,status:412,auditlog,chain"
SecRule REQUEST_LINE "post .*/wp-login"

This unfortunately doesn’t prevent “distributed” attacks in which many different IP addresses submit different password guesses, but it will help in many cases.

Summary

So: if you use WordPress yourself, be sure to update. And if you provide blog hosting services for others, consider adding the mod_security rules to your Apache server.

Although all WordPress users should upgrade right away, we’ve added security rules to our servers to protect our Web hosting customers who haven’t yet upgraded. Other people may find the rules useful if they use mod_security on Apache Web servers. The rest of this post contains more technical details.

The first thing to be concerned about is the problem fixed in version 2.8.4. Earlier versions allowed strangers to repeatedly reset the administrator’s password to a random string of text. This doesn’t allow the stranger to gain access to your blog, but it sure is annoying.

The exploit works because PHP interprets HTTP parameters that end with two square brackets, like this:

key[]=

… as an array, and WordPress didn’t check for that possibility. These mod_security rules block any parameters to wp-login.php that contain square brackets:

SecRule REQUEST_FILENAME "/wp-login\.php$" "deny,status:412,auditlog,chain"
SecRule ARGS_NAMES "\["

So that will prevent strangers from resetting passwords.

In addition to that, we've discovered something else interesting. Earlier versions of WordPress (all versions before 2.8.3) seem to contain more of a security problem than previously thought.

The release announcements for versions 2.8.1 and 2.8.3 said "admin pages added by certain plugins could be viewed by unprivileged users, resulting in information being leaked", then "I missed some places when fixing the privilege escalation issues for 2.8.1".

Allowing unprivileged users to see information they shouldn't see is undesirable, but again, it doesn't seem to allow strangers to take over your blog.

Unfortunately, we've found that the bug does actually allow clever unprivileged attackers to change some of the blog settings in version 2.8.2 and earlier. And a really clever attacker can leverage this into a "remote code exploit" by taking advantage of a strange PHP feature called "Complex (curly) syntax". The blogs of two of our customers were hijacked today by an exploit that does exactly this.

We won't go into full details yet, because it doesn't seem that the vulnerability has been published elsewhere (we've contacted the WordPress folks in case they weren't aware). But we will say that the attack can't succeed unless the server allows people to request "wp-admin" URLs with two consecutive slashes, so the following mod_security rule blocks it:

SecRule REQUEST_FILENAME "wp-admin.*//" "deny,status:412,auditlog"

So there are several reasons to make sure you've upgraded your own blog to 2.8.4. (The WordPress "automatic upgrade" feature usually makes this easy.) And if you run a server that hosts WordPress blogs, consider adding the two mod_security rules mentioned above to protect your users.

Update September 6: The new security problem we mentioned above is being widely discussed in posts like How to Keep WordPress Secure and Old WordPress Versions Under Attack. Although our customers have been protected against this particular new attack since August 12, as described above, you should certainly still upgrade your copy of WordPress to protect against other attacks.

Final update: The technical details of the problem in WordPress 2.8.2 and earlier were that WordPress was not trimming double slashes from paths like /wp-admin//options-permalink.php, which allowed anyone to load it. In addition, the “options-permalink.php” file was missing the if ( !current_user_can('manage_options') ) check that prevented any registered user from changing the permalink structure by accessing such a URL. So anyone who could register could also change the permalink structure. And a third issue meant that changing the permalink structure to include the bizarre text ({${eval(base64_decode($_SERVER[HTTP_EXECCODE]))}}|.+) caused WordPress to run remote code. The WordPress folks fixed these three bugs here, here, and here, respectively.