Between 7:00 and 7:45 PM Pacific time Thursday night (March 11), we received two reports of slow or nonexistent network connections to sites on our servers.
Our automated monitoring systems didn’t detect any general problems, so the majority of customers were certainly unaffected — but we suspect that one of the “Internet backbones” between the affected customers and our data center had high packet loss during that period.
Both customers reported that the problem resolved itself by 7:45, and we haven’t received similar reports since, so there does not appear to be be an ongoing problem. We’ll continue to monitor it closely.
The “calculon” Web server will be restarted at 11 PM Pacific time tonight (February 19). This will cause a five-minute interruption of Web and e-mail service for customers on that server.
Other servers will not be affected, and incoming mail will only be delayed, not lost.
We apologize for the problem and for the short notice: the restart is necessary to replace a disk in the RAID array.
Update 11:03 PM Pacific time: The restart was completed with less than 3 minutes “downtime”.
The “bender” Web server experienced intermittently high load between about 7:40 and 10:15 AM Pacific time this morning, February 18. This resulted in slow or even inaccessible Web sites on that server. (Some e-mail was also delayed before being properly delivered.) Other servers were not affected.
This server had similar high load symptoms (but much more briefly) earlier this week. We took some steps to reduce the load then, but it appears those weren’t sufficient. We’re now taking much stronger action to ensure that this does not happen again.
We sincerely apologize to customers affected by this problem. We don’t consider it normal or acceptable, and we will make sure this isn’t a recurring issue.
After upgrading our systems to PHP 5.2.6, we received reports of an incompatibility with Joomla. Some URLs do not work when Joomla is configured to use “Search Engine Friendly URLs”, but to not have “Use Apache mod_rewrite” turned on.
We’ve investigated this, and it’s caused by Joomla assuming that PHP has a bug that makes it work incorrectly, when in fact it’s supposed to work differently (and is clearly documented to work differently). Older versions of PHP had this bug, but the new version doesn’t.
To help our customers work around this, we’ve “patched” PHP to intentionally reintroduce the old bug for now, thus keeping it “compatible” with Joomla. If you were having trouble with Joomla’s “Search Engine Friendly URLs”, it should be fixed.
We’ll provide more technical details (and a more robust long-term solution) in the near future.
Update: We’ve also reported this problem to the Joomla developers and suggested a solution.
Due to software updates on our servers, Web hosting customers will experience about seven minutes of scheduled maintenance downtime between 11 PM and 1 AM Pacific time starting on one of the following nights, depending on which server your site is on:
- Thursday, February 4 (servers beginning with the letter “L”, such as “lrrr”)
- Friday, February 5 (all other servers beginning with letters “F-Z”, such as “farnsworth”)
- Saturday, February 6 (servers beginning with letters “A-E”, such as “amy”)
Read the rest of this entry »
Starting at about 1:30 PM (Pacific time) today, some of our internal database systems had an unexpected failure. This lead to problems with our control panel (“My Account”), support site, and blog. Some customers may have also had problems with some aspects of Webmail (in particular, with the address book).
No customer data or Web sites were affected, and no e-mail was lost.
All systems are running again, so no one should see any problems — please let us know if you do! Some things are running on backup systems, so we’re working on finishing up the fixes and restoring everything to its normal status.
This was an unplanned and fairly horrible (and embarrassing) problem. This is the first time our own account management database has completely failed in the 10 years we’ve been providing Web hosting service. Obviously, we consider this sort of thing to be unacceptable, and we sincerely apologize for any inconvenience caused.
Read the rest of this entry »
Between 10:00 PM and 11:59 PM Pacific time this Sunday January 3, all our hosting servers will be restarted. As a result, Web site service and the ability to read incoming e-mail will be unavailable for approximately five minutes at some point during this maintenance “window”.
Read the rest of this entry »
Our own www.tigertech.net Web site and “My Account” control panel will be unavailable for approximately 15 minutes beginning at 9 PM Pacific time tonight (December 2). We’ll be replacing and upgrading some hardware.
Customer Web sites and e-mail will not be affected at all.
Customers on the “amy” server experienced a nine minute interruption in Web site and e-mail service between 5:58 and 6:07 PM Pacific time today (November 16).
Customers on other servers were not affected.
Read the rest of this entry »
Our “bender” server encountered had an extremely high load starting at about 1:36 PM (Pacific time) today, and lasting until about 2:00 PM. Due to unusual circumstances, a runaway customer script caused the RAID disk array to become overloaded with writes, causing scripts that write to the disk to run slowly. The server may have seemed slow or unresponsive to customers at some point. We don’t consider this type of event to be acceptable, and apologize for the inconvenience.
We fixed the problem, and modified the server’s settings to hopefully prevent a recurrence. The changes were propagated to all similar hosting servers to protect them as well.
Read the rest of this entry »
