We’re receiving reports of network connectivity problems from a couple of customers using the “Global Crossing” Internet backbone to reach our primary data center, although most customers are unaffected. We’re investigating this issue.
Update 12:35 PM: Our upstream provider reports that an 8 minute network interruption for some connections, beginning at 11:11 AM Pacific time, was caused by a router failure at Global Crossing. The problem has been resolved.
We’ve recently heard from several customers who have received what appears to be a domain name renewal invoice from a company called “Domain Registry of America”.
These “invoices” are a scam. Domain Registry of America is unrelated to our company, and has been cited by the FTC for “deceptive conduct”.
If you look closely at the “invoices”, they actually say something like “This notice is not a bill, rather an easy means of payment should you decide to renew your domain with us.” However, that small print is easy to miss.
We have a page about Domain Registry of America scams with much more information. We encourage you to make sure that whoever pays your invoices is aware of it.
We recently added the Spamhaus Domain Block List (dbl.spamhaus.org) to our spam filters.
The Domain Block List is an extremely reliable list of domain names that are used only in spam. Blocking most mail that advertises these domain names improves our spam filtering: we’re now blocking about 1% more spam as a result.
That may not sound like much, but it represents about 150 more blocked spam messages per year for each customer. (We block an average of over 15,000 spam messages per year per customer.)
Between 7:00 and 7:45 PM Pacific time Thursday night (March 11), we received two reports of slow or nonexistent network connections to sites on our servers.
Our automated monitoring systems didn’t detect any general problems, so the majority of customers were certainly unaffected — but we suspect that one of the “Internet backbones” between the affected customers and our data center had high packet loss during that period.
Both customers reported that the problem resolved itself by 7:45, and we haven’t received similar reports since, so there does not appear to be be an ongoing problem. We’ll continue to monitor it closely.
If you use a MySQL database with large tables, it’s possible to accidentally run queries that try to sort millions of rows (usually through some kind of programming error, such as an “unconstrained table join”).
Those runaway queries can slow down the MySQL server for many minutes on end, causing performance problems.
To prevent the worst of that, we’ve set the max_join_size setting to 1,000,000 on our MySQL servers.
We keep coming across WordPress customer sites that have hurt their performance by switching from the “WP Super Cache” plugin we recommend to a newer plugin named “W3 Total Cache”. Unfortunately, their site often ends up being far slower after switching to W3 Total Cache.
If you care about the performance of your site, please stick with WP Super Cache unless you have a very good reason to switch. It works, and it works well.
Some people tell us that W3 Total Cache works just as well if it’s properly configured, and they may well be right — but it seems like it’s difficult to configure properly. Our experience is showing that it’s easy to get wrong, and performance ends up suffering. WP Super Cache makes it easy to get great performance.
The “calculon” Web server will be restarted at 11 PM Pacific time tonight (February 19). This will cause a five-minute interruption of Web and e-mail service for customers on that server.
Other servers will not be affected, and incoming mail will only be delayed, not lost.
We apologize for the problem and for the short notice: the restart is necessary to replace a disk in the RAID array.
Update 11:03 PM Pacific time: The restart was completed with less than 3 minutes “downtime”.
The “bender” Web server experienced intermittently high load between about 7:40 and 10:15 AM Pacific time this morning, February 18. This resulted in slow or even inaccessible Web sites on that server. (Some e-mail was also delayed before being properly delivered.) Other servers were not affected.
This server had similar high load symptoms (but much more briefly) earlier this week. We took some steps to reduce the load then, but it appears those weren’t sufficient. We’re now taking much stronger action to ensure that this does not happen again.
We sincerely apologize to customers affected by this problem. We don’t consider it normal or acceptable, and we will make sure this isn’t a recurring issue.
Our business offices will be closed on Monday, February 15 to observe the US legal holiday. As always, we’ll provide same-day support for time-sensitive issues via our ticket and e-mail systems. However, questions that aren’t time-sensitive (including most billing matters) may not be answered until the next day, and telephone support (via callbacks) will be available only for urgent problems.
After upgrading our systems to PHP 5.2.6, we received reports of an incompatibility with Joomla. Some URLs do not work when Joomla is configured to use “Search Engine Friendly URLs”, but to not have “Use Apache mod_rewrite” turned on.
We’ve investigated this, and it’s caused by Joomla assuming that PHP has a bug that makes it work incorrectly, when in fact it’s supposed to work differently (and is clearly documented to work differently). Older versions of PHP had this bug, but the new version doesn’t.
To help our customers work around this, we’ve “patched” PHP to intentionally reintroduce the old bug for now, thus keeping it “compatible” with Joomla. If you were having trouble with Joomla’s “Search Engine Friendly URLs”, it should be fixed.
We’ll provide more technical details (and a more robust long-term solution) in the near future.
Update: We’ve also reported this problem to the Joomla developers and suggested a solution.
