Tonight at 11 PM Pacific time (2 AM Eastern time May 10) we’ll be performing brief scheduled maintenance on our mail servers. (We’ll be adding more RAM and adding more disk space to make sure that our mail servers continue to keep up with the growth in our service.) This requires restarting, which takes about five minutes, so you will see a brief period of about five minutes where you are unable to connect to our mail servers. No mail will be lost, of course; it will be queued and available after the maintenance.
We apologize for the inconvenience this causes. We schedule this kind of maintenance for late Saturday night/early Sunday morning (the least busy time of the week) to minimize the impact.
We’ve installed several security updates recently. We’ve updated PHP 4, PHP 5, the ClamAV antivirus scanner, and some XFree86 libraries. In addition, we’ve updated our own blog to use WordPress 2.2 — if you use WordPress, make sure you’ve done the same.
Read the rest of this entry »
We’ve updated the default version of Ruby on Rails on our servers to version 1.2.3.
Read the rest of this entry »
A couple of times in the last week, we’ve seen one of our MySQL database servers have an unusually high number of connections. That’s a serious issue: If there are too many connections to a MySQL server, customer scripts won’t be able to connect to a database, so we’ve spent some time looking at the cause and fixing it.
Read the rest of this entry »
One of the features of our service is the industrial-strength Mailman mailing list manager. Mailman is a very good program in some ways (it’s built like a tank and reliably handles very large volumes of list mail, and it removes much of the drudgery of managing large lists), but it has a couple of undesirable “features”.
The most obvious is that the interface is terribly ugly (the Mailman developers are working on a big improvement to this, thankfully; just so it’s clear, we didn’t create the program, and we’re as horrified by the circa-1996 appearance as everyone else). Another problem with the program, though, is the option for “monthly password reminders”. This is a design flaw that’s being removed from Mailman, and although most of the lists on our servers don’t use password reminders, customers who do should probably turn them off now in preparation for that change.
Read the rest of this entry »
The “farnsworth” Web server locked up and needed restarting again last night at about 9:04 PM Pacific time, causing another short outage for some customers. (A similar problem happened Monday night.) To make sure this doesn’t happen again, we’ll be replacing the entire server (switching it with a spare server) at about 11 PM (Pacific) tonight, which will result in about 5 minutes of downtime.
We’re also taking this opportunity to upgrade the hardware on one of our mail servers to allow for future growth; customers (even those with accounts on other Web servers besides the farnsworth server) may see a short (approximately 5 minute) interruption in their ability to retrieve e-mail between 11 PM and midnight.
We apologize for any inconvenience this causes — as always, we’re committed to the highest possible levels of reliability.
Some of the posts on our blog mention specific servers. You’ll occasionally see things like “The web14 server will be rebooted at 11 PM”, “mail sent from the web01 server was delayed”, or “more memory has been added to the web10 server”. Your question, quite naturally, is “How do I know if they’re talking about the server that has my account?”
Read the rest of this entry »
One of our Web servers (the “farnsworth” server) stopped responding at 7:07 PM Pacific time today, and needed to be forcibly restarted. This resulted in a Web server and FTP server outage of about 15 minutes for some customers, although most sites were unaffected.
After being restarted, the server is responding properly, but still showing a problem with one of the disks in its RAID array. Because of that, we plan to replace the disk to prevent future problems, meaning we’ll restart that server again later tonight (after 11 PM Pacific time).
We apologize to all customers affected; we strive hard to avoid this kind of problem.
Our business offices will be closed on Monday, May 28 to observe the US legal holiday. As always, we’ll provide same-day support for time-sensitive issues via our ticket and e-mail systems. However, questions that aren’t time-sensitive (including most billing matters) may not be answered until May 29, and telephone support (via callbacks) will be available only for urgent problems.
We’ve updated PHP 5 on our servers to cover sixteen recently identified security issues. This only affects customers who have chosen to use PHP 5 — but since this upgrade only fixes security bugs, even those customers shouldn’t notice any changes.
Read the rest of this entry »
