But recently we’ve seen a number of our customers getting hammered by a ton of requests from FeedBurner. Usually the request is of this form:

/somepost?utm_source=feedburner&utm_medium=feed&utm_campaign=SomeCampaignString

We’ve also seen FeedBurner going crazy and making thousands of duplicate requests. One of the sites we host has gotten 45,000 simple status requests (HTTP “HEAD” requests) from FeedBurner today, for no good reason that we can see.

Unfortunately, the default rules of WP Super Cache prevent it from caching any request with a query that contains an equal sign. So all of these requests are being unnecessarily run freshly each time, rather than being served from the cache.

There’s an easy fix to this. Open your Web site’s .htaccess file. Look for the section of lines for WP Super Cache, and find the line which tests %{QUERY_STRING}. Insert this new line of text immediately above the existing line:

RewriteCond %{QUERY_STRING} ^utm_source=(feedburner|twitterfeed) [OR]

The new line (ending with [OR]) must come before the existing %{QUERY_STRING} line. After inserting, the two lines should look exactly like this:

RewriteCond %{QUERY_STRING} ^utm_source=(feedburner|twitterfeed) [OR]
RewriteCond %{QUERY_STRING} !.*=.*

There are two very similar blocks right next to each other in the .htaccess file. Be sure to add the new line to the same place in each block.

That’s the only change you’ll have to make! WP Super Cache will now be able to cache requests for normal pages that come from FeedBurner or Twitterfeed. If your Web site was being abused by FeedBurner (or Twitterfeed), you should see a definite improvement.

In some cases, you might be storing data that’s not quite so important. And if it means your application can run much faster, you might be willing to risk a very small chance of data loss. That’s where MySQL’s “INSERT DELAYED” statement, which works with MyISAM table types (but not InnoDB tables), can be useful. (Tables are created as type MyISAM by default, so most tables are eligible to benefit from this tip.)

Adding the word “DELAYED” to your statement tells MySQL to remember the data to be added and return immediately to your application. MySQL will then write the data as soon as the database isn’t busy. This lets your insertion happen (effectively) immediately, and reduces the load on the database (and server). Using this technique can give your application a huge performance gain.

Since the record is not written immediately, there is a very small chance that the data will be lost before it’s written to the disk. However, the odds of this happening are very small. It would only happen if MySQL crashed before it had a moment of idle time to write out the record, if the server lost power, or if some similar unexpected event happened.

Using “INSERT DELAYED” is recommended for applications that do not absolutely depend upon records being immediately written. Any data which is only referred to at a later time (rather than on-demand) or in a summary fashion is a good candidate for using “INSERT DELAYED”. Some examples are logging the IP address of each visitor, or tracking ad impressions for each page viewed on your Web site. Be sure to read the official documentation for full details and additional considerations.

If you want to use “INSERT DELAYED”, first check the documentation to see if your application already supports it. If it does not, you’d need to modify the application. If you have any questions about whether a particular database table is a good candidate for using “INSERT DELAYED”, just ask us and we’ll take a look.

That advice makes sense if you have a sudden surge of traffic to a single page. However, if your site is generally very busy across all pages (for example, if you have an archive of hundreds or thousands of posts that are constantly being indexed by search engines), we’ve found that you should do the opposite to improve performance: set it much higher. We recommend setting it to 172800 seconds (which is 48 hours). This can cut your CPU usage in half, which will speed up your site.

The reason for this is that when WP Super Cache creates a cached page, it wants to make sure that those pages don’t build up forever. Every ten minutes or so, it looks through them all and deletes any that are older than the “expire time”.

On some servers that use a network file system called “NFS”, looking through a large number of files causes performance problems. That’s why the WP Super Cache author recommends making them expire quickly: it reduces the number of files it has to examine each time.

On our servers, we don’t use NFS and looking through lots of files does not cause a performance problem. Leaving the files for a longer time is safe and increases the chance that a page will already be cached when it’s needed.

If you’re a Tiger Technologies customer who makes this change and you want to see how it affects the CPU usage, just let us know and we can provide you with details.

Although all WordPress users should upgrade, we’ve added security rules to our servers to protect our Web hosting customers who haven’t yet upgraded. Other people may find the rules useful if they use mod_security on Apache Web servers. The rest of this post contains more technical details.

Surprising Apache script behavior

One of the fixes in WordPress 2.8.6 compensates for a peculiar and surprising feature (or bug, if you prefer) in the Apache Web server: in many cases, it will run a file with a name like “image.php.jpeg” as a PHP script.

Some software (like WordPress 2.8.5 and earlier) can be configured to allow strangers to upload JPEG and other files to your site — but it checks only the “.jpeg” file extension to see if the uploaded file is safe. So a “hacker” could upload a malicious PHP script named “image.php.jpeg”, then “view the image” in a Web browser… but the server would actually run the PHP script.

Because of that, we’ve added a mod_security rule that prevents site visitors from requesting certain file extensions that include “.php.” in their name. (There are other possible solutions, but testing has shown this is the least intrusive to our existing customers.)

Here’s a mod_security rule that accomplishes that (adjust the extensions to suit your taste; these are the WordPress 2.8.5 allowed extensions):

SecRule REQUEST_FILENAME "\.php[456]?\.(asf|asx|avi|bmp|gif|ico|jpe|jpeg|jpg|png|tif|tiff|wax|wmv|wmx)$" "deny,status:412,auditlog"

WordPress brute force attacks

Another recently reported attack against WordPress (which is unrelated to version 2.8.6 in particular) involves “brute force” attempts to guess the administrator password. The proper solution is to choose a good password for your blog (not a word from the dictionary!), but some people don’t do that.

To reduce the risk of successful attacks against our customers, we limit each IP address to 25 “wp-login.php” attempts within a five minute period. Here’s how you can do that with mod_security:

SecAction phase:1,initcol:ip=%{REMOTE_ADDR},nolog
SecRule REQUEST_LINE "post .*/wp-login" "nolog,phase:1,setvar:ip.wordpress_login=+1,deprecatevar:ip.wordpress_login=5/60"
SecRule IP:WORDPRESS_LOGIN "@gt 25" "deny,status:412,auditlog,chain"
SecRule REQUEST_LINE "post .*/wp-login"

This unfortunately doesn’t prevent “distributed” attacks in which many different IP addresses submit different password guesses, but it will help in many cases.

Summary

So: if you use WordPress yourself, be sure to update. And if you provide blog hosting services for others, consider adding the mod_security rules to your Apache server.

Although all WordPress users should upgrade right away, we’ve added security rules to our servers to protect our Web hosting customers who haven’t yet upgraded. Other people may find the rules useful if they use mod_security on Apache Web servers. The rest of this post contains more technical details.

The first thing to be concerned about is the problem fixed in version 2.8.4. Earlier versions allowed strangers to repeatedly reset the administrator’s password to a random string of text. This doesn’t allow the stranger to gain access to your blog, but it sure is annoying.

The exploit works because PHP interprets HTTP parameters that end with two square brackets, like this:

key[]=

… as an array, and WordPress didn’t check for that possibility. These mod_security rules block any parameters to wp-login.php that contain square brackets:

SecRule REQUEST_FILENAME "/wp-login\.php$" "deny,status:412,auditlog,chain"
SecRule ARGS_NAMES "\["

So that will prevent strangers from resetting passwords.

In addition to that, we’ve discovered something else interesting. Earlier versions of WordPress (all versions before 2.8.3) seem to contain more of a security problem than previously thought.

The release announcements for versions 2.8.1 and 2.8.3 said “admin pages added by certain plugins could be viewed by unprivileged users, resulting in information being leaked”, then “I missed some places when fixing the privilege escalation issues for 2.8.1″.

Allowing unprivileged users to see information they shouldn’t see is undesirable, but again, it doesn’t seem to allow strangers to take over your blog.

Unfortunately, we’ve found that the bug does actually allow clever unprivileged attackers to change some of the blog settings in version 2.8.2 and earlier. And a really clever attacker can leverage this into a “remote code exploit” by taking advantage of a strange PHP feature called “Complex (curly) syntax“. The blogs of two of our customers were hijacked today by an exploit that does exactly this.

We won’t go into full details yet, because it doesn’t seem that the vulnerability has been published elsewhere (we’ve contacted the WordPress folks in case they weren’t aware). But we will say that the attack can’t succeed unless the server allows people to request “wp-admin” URLs with two consecutive slashes, so the following mod_security rule blocks it:

SecRule REQUEST_FILENAME "wp-admin.*//" "deny,status:412,auditlog"

So there are several reasons to make sure you’ve upgraded your own blog to 2.8.4. (The WordPress “automatic upgrade” feature usually makes this easy.) And if you run a server that hosts WordPress blogs, consider adding the two mod_security rules mentioned above to protect your users.

Update September 6: The new security problem we mentioned above is being widely discussed in posts like How to Keep WordPress Secure and Old WordPress Versions Under Attack. Although our customers have been protected against this particular new attack since August 12, as described above, you should certainly still upgrade your copy of WordPress to protect against other attacks.

We have a support page available which walks you through setup using AutoDiscover.

Although all WordPress users should upgrade right away, we’ve also added a security rule to our servers to try and protect our customers who haven’t yet upgraded. Other people may also find the security rule useful if they use mod_security on Apache Web servers. The rest of this post contains more technical details.

So what’s the security problem with WordPress 2.5? Well, if your blog allows strangers to register, the vulnerability allows evildoers to create a new unprivileged username like “admin0″, then use the cookie from that username to login as the privileged username “admin”.

This works because WordPress creates a login cookie with a value like this for “admin0″:

admin0|1209331453|2a771ff005c67b2aaa0a872aaa213f39

The first part is the username, the second part is the cookie’s expiration time, and the third part is an MD5 hash of the concatenated username, expiration, and some secret text.

The trouble is that this cookie is also valid:

admin|01209331453|2a771ff005c67b2aaa0a872aaa213f39

We simply moved the “0″ from the end of “admin” to the beginning of the expiration. Although this changes the username, it doesn’t change the MD5 hash! That’s because the text that the hash is based on doesn’t contain a separator such as a pipe character, as it should. In either case, it’s just hashing “admin01209331453″ (plus the same secret text), and the identical hash is valid for both the privileged and unprivileged users. A nasty bug indeed.

You don’t necessarily need to use “0″ on the end of the username, either; other digits can be made to work. They increase the expiration time of the cookie, but that doesn’t make it invalid.

So this vulnerability allows people to create a WordPress username ending in one or more digits, then use the cookie from that username to login as another username without the digits on the end.

Here’s where mod_security can be useful. It’s a great defensive tool that we’ve been using more and more to protect our customers from this kind of attack. If it’s possible to identify something that’s different about an “evil” page request (compared to a “good” page request), you can probably block the evil request using mod_security.

In this case, one thing immediately stands out: a “good” request always has a 10 digit expiration time in the cookie, but a “bad” request always has 11 or more digits. If we can block WordPress cookies that contain 11 or more digit expirations, that should block the evil request.

This rule for mod_security 1.x does the trick:

SecFilterSelective HTTP_Cookie "wordpress_[a-f0-9]{32}=[^\|]+\|[0-9]{11,}\|[a-f0-9]{32}"

And here’s the same rule for mod_security 2.x:

SecRule REQUEST_HEADERS:Cookie "wordpress_[a-f0-9]{32}=[^\|]+\|[0-9]{11,}\|[a-f0-9]{32}"

This looks for a cookie value containing 11 or more digits between the pipe symbols. This rule successfully blocks a test attack we created, and does not appear to give false positives (we’ve had it in place on servers that have served several million pages over the last few hours, with no matches beyond our tests).

Just so it’s clear, our customers don’t need to use these rules, because they’re already on all our servers — but we hope they’re useful to someone else.

Of course, if you know of any other attack vectors, please post a comment so we can improve the rules.

However, thread view has a potential downside: it you have several active threads going with several messages each, new messages can sometimes appear on the second page of the incoming mail screens, instead of the first page.

That’s not a problem if you’re expecting it. However, since we introduced the new Webmail system, we’ve had several complaints from customers who accidentally clicked “Switch to Thread View” without realizing what it does, then thought some of their incoming mail was missing because they aren’t used to looking for new mail on other pages. Since thread view is “remembered” even after you logout and login again, this caused some people a great deal of heartache.

From our logs, we’ve found that very few people actually use thread view. Because it seems to cause frequent problems and few people use it, we’ve made it an optional feature instead of being always enabled.

If (like most people) you don’t use thread view, you don’t need to do anything. If do you want to use thread view, it’s still available: just click “Preferences”, then click “Display Preferences”, then change “Show ‘Thread View’ Link” to “Yes”.

Anyway, it turns out that Apple convinced some of you to take leave of your financial senses, too, and you’ve been asking us how to set up your iPhone to read your e-mail. So we’ve spent many hours voiding the warranty on our phone, getting it to the point where we could extract detailed screen shots showing exactly how to set up iPhone mail. If you have an iPhone, give it a try! Our servers handle iPhone e-mail connections just fine — and the connections are fully encrypted by default, making sure your e-mail and passwords stay secure as you roam the world on strangers’ WiFi networks.

The most obvious is that the interface is terribly ugly (the Mailman developers are working on a big improvement to this, thankfully; just so it’s clear, we didn’t create the program, and we’re as horrified by the circa-1996 appearance as everyone else). Another problem with the program, though, is the option for “monthly password reminders”. This is a design flaw that’s being removed from Mailman, and although most of the lists on our servers don’t use password reminders, customers who do should probably turn them off now in preparation for that change.

So what’s the issue? The main problem is that Mailman sends people’s personal passwords over the Internet via plain text e-mail, which is about as secure as sending them around the world on a postcard. One of the most basic password security rules for users is “memorize a password instead of writing it down anywhere”, and Mailman completely breaks that rule. And one of the most basic rules for secure software designers is “store passwords only in encrypted form so that even the program author and system administrator can’t tell what they originally were, so that if hackers break into the system they can’t tell either”. Mailman breaks this rule, too — it has to, in order to be able to send the monthly password reminders. The authors of Mailman attempted to solve this problem by adding a note to the subscription page saying “do not use a valuable password”, but that goes against human nature; some people will inevitably enter the same password they use for online banking, etc.

There’s another problem with password reminders, too, which is that if you have more than one Mailman list, the password reminders are sent from the e-mail address of a seemingly random one of those lists, even if the recipient isn’t subscribed to that particular list. Not a security problem, but certainly annoying.

Anyway, bottom line: monthly password reminders are a bad idea and don’t work properly anyway. The good news is that the Mailman developers have completely removed password reminders and plaintext passwords from the next version of Mailman, which we’ll be using when it becomes available. If your list uses monthly password reminders, they’re going to go away, hopefully soon. (In fact, we consider the password reminders such a source of problems that if the new version of Mailman is substantially delayed, we may end up preemptively disabling them for all lists on our servers.)

This actually won’t affect most of our customers, because the Mailman setup wizard in our control panel hasn’t turned on the password reminders option for new lists for a long time now. However, a few older lists had it enabled by default, and it’s still possible for customers to manually enable it by clicking one of the obscure options in the Mailman administrative interface.

If your list uses password reminders, we recommend turning them off now:

1. Login to the administration page for your list
2. Scroll down to the “Send monthly password reminders?” setting and change it to “No”
3. Scroll to the bottom of the page and click “Submit Your Changes”.

Doing this makes the list more secure for your subscribers, and also makes sure you’re not surprised when password reminders stop by themselves.

If you have trouble disabling the reminders, just contact us and we’ll be glad to do it for you.