MySQL 5 Upgrade (Saturday August 4th)

This coming Saturday (August 4th), we’ll be upgrading the MySQL database software on all our servers from version 4.1.11 to version 5.0.32 (plus appropriate security updates). MySQL 5 adds many features that customers have requested, and some Web site scripts now require it.

The upgrade will take place between 11:00 PM and 11:59 PM Pacific time (the times will be slightly different for each server). We’ll be making a full backup of all databases on each server before the upgrade, and as a result, customers should expect MySQL to be unavailable for about 15 minutes during this period. In addition, large databases may be slow for several minutes after the upgrade, because MySQL automatically converts databases to the new version format the first time they are accessed, which can be time consuming.

If your site doesn’t use MySQL databases, it won’t be affected at all. If it does, you almost certainly won’t see any effects other than the 15 minute outage. That said, we do recommend checking your scripts for MySQL 5 compatibility.

Read the rest of this entry »

Mailman message limits doubled

We’ve been increasing our mail server capacity lately, and as a result we’ve doubled the number of Mailman messages that hosting customers can send each month. Our Silver plan now includes up to 60,000 deliveries, our Gold plan includes 120,000, and our Platinum plan includes 300,000.

More details about what’s included on each plan are always available on our plans page.

Elzar server temporarily unavailable (resolved)

Starting around 9:03 PM (Pacific time) tonight, our elzar server had an extremely high load placed upon it. The result was that many users may have had problems connecting to Web sites running on elzar. The problem occurred intermittently for about 30 minutes, at which point we managed to restore normal service.

Please be assured that we do our best to make sure that our server loads always stay within reasonable limits to avoid just this kind of problem. The load spike was unexpected and was well outside of the generous safety limits that we keep on each server. We will continue to monitor the situation, and will take corrective or preventive action if appropriate. We appreciate your business, and apologize for any inconvenience.

4th of July holiday hours

Our business offices will be closed on Wednesday, July 4 to observe the US legal holiday. As always, we’ll provide same-day support for time-sensitive issues via our ticket and e-mail systems. However, questions that aren’t time-sensitive (including most billing matters) may not be answered until the next day, and telephone support (via callbacks) will be available only for urgent problems.

Temporary routing problem (resolved)

Between 4:33 and 4:41 PM Pacific time, we experienced a short-lived problem where users who reach our servers via an “Internet backbone” called Global Crossing (including Comcast and Charter cable customers) were unable to connect. Other users weren’t affected.

The problem lasted for less than ten minutes, and everything is now operating normally.

E-mail, zapp, lrrr, servers temporarily unavailable (resolved)

Due to a failure of the power distribution unit (essentially a fancy power strip) in one of the cabinets at our data center, the following services became unavailable at 05:52 AM Pacific time:

(Other Web servers are not affected.) A data center technician is replacing the power unit in that cabinet and all systems should be be back online within 15 minutes; we’ll update this post when that happens.

Update: The faulty hardware has been completely replaced. All servers are back online and functioning normally, and all queued e-mail has been delivered and is available for retrieval. The total outage for these servers was from 05:52 AM to 06:15 AM (Pacific time).

In addition, the FTP service on the “zapp” server was not fully working after it was restarted, so FTP publishing on that server was unavailable until shortly after 7:00 AM. This has been corrected (and the underlying problem that could cause incorrect startup was fixed).

We sincerely apologize to customers affected by this outage. This kind of issue has happened to us only once before in the last seven years (and that was with a different brand of power unit). Since the replacement power unit is brand new, we don’t expect the problem to recur.