Between 11:00 PM and 11:59 PM Pacific time on Saturday August 11, all Tiger Technologies Web hosting servers will be restarted. As a result, customer Web sites, as well as the Tiger Technologies Web site, will be unavailable for approximately five minutes. E-mail service will not be affected.
This brief maintenance is necessary for two reasons. First, we’re upgrading the operating system “Linux kernel” to a newer version for security reasons. Secondly, we’re adding more memory to our hosting servers, so that each server will have 4 GB of RAM instead of the current 2 GB.
Read the rest of this entry »
We’ve finished upgrading all our Web hosting servers to use MySQL 5.0.32, as we explained in this earlier post. The total time that MySQL databases were unavailable averaged about ten minutes per server. Again, we regret any downtime — this was unavoidable for such a major upgrade.
You should not notice any difference in how your site operates. In the unlikely event you have any problems, please contact us by opening a ticket so we can help.
Since about 9:00 AM (Pacific time) this morning, we’ve been seeing network routing problems to some destinations on the Internet that use the “xo.net” backbone. For some customers, this will have the effect of making any access to your web site extremely slow — it may even be so slow as to seem completely non-responsive. Most customers will have no problems.
Our data center technicians are working on this problem. We’ll update this post as soon as the issue is resolved.
Update: This issue was resolved at approximately 10:20 AM, and all systems are operating normally.
This coming Saturday (August 4th), we’ll be upgrading the MySQL database software on all our servers from version 4.1.11 to version 5.0.32 (plus appropriate security updates). MySQL 5 adds many features that customers have requested, and some Web site scripts now require it.
The upgrade will take place between 11:00 PM and 11:59 PM Pacific time (the times will be slightly different for each server). We’ll be making a full backup of all databases on each server before the upgrade, and as a result, customers should expect MySQL to be unavailable for about 15 minutes during this period. In addition, large databases may be slow for several minutes after the upgrade, because MySQL automatically converts databases to the new version format the first time they are accessed, which can be time consuming.
If your site doesn’t use MySQL databases, it won’t be affected at all. If it does, you almost certainly won’t see any effects other than the 15 minute outage. That said, we do recommend checking your scripts for MySQL 5 compatibility.
Read the rest of this entry »
Starting around 9:03 PM (Pacific time) tonight, our elzar server had an extremely high load placed upon it. The result was that many users may have had problems connecting to Web sites running on elzar. The problem occurred intermittently for about 30 minutes, at which point we managed to restore normal service.
Please be assured that we do our best to make sure that our server loads always stay within reasonable limits to avoid just this kind of problem. The load spike was unexpected and was well outside of the generous safety limits that we keep on each server. We will continue to monitor the situation, and will take corrective or preventive action if appropriate. We appreciate your business, and apologize for any inconvenience.
Between 4:33 and 4:41 PM Pacific time, we experienced a short-lived problem where users who reach our servers via an “Internet backbone” called Global Crossing (including Comcast and Charter cable customers) were unable to connect. Other users weren’t affected.
The problem lasted for less than ten minutes, and everything is now operating normally.
Due to a failure of the power distribution unit (essentially a fancy power strip) in one of the cabinets at our data center, the following services became unavailable at 05:52 AM Pacific time:
(Other Web servers are not affected.) A data center technician is replacing the power unit in that cabinet and all systems should be be back online within 15 minutes; we’ll update this post when that happens.
Update: The faulty hardware has been completely replaced. All servers are back online and functioning normally, and all queued e-mail has been delivered and is available for retrieval. The total outage for these servers was from 05:52 AM to 06:15 AM (Pacific time).
In addition, the FTP service on the “zapp” server was not fully working after it was restarted, so FTP publishing on that server was unavailable until shortly after 7:00 AM. This has been corrected (and the underlying problem that could cause incorrect startup was fixed).
We sincerely apologize to customers affected by this outage. This kind of issue has happened to us only once before in the last seven years (and that was with a different brand of power unit). Since the replacement power unit is brand new, we don’t expect the problem to recur.
The “elzar” Web server stopped responding a few minutes ago under a heavy load on the MySQL database server, and had to be restarted. This resulted in an interruption of service for Web sites on that server.
We apologize for this problem; we’ll be investigating the issue further and monitoring the server closely to make sure it doesn’t recur.
Update 10:00 PM: The NFS network connection between ftp.tigertech.net and elzar wasn’t working properly even after the Web server was restarted, causing additional problems for customers publishing files. This problem has also been corrected.
Tonight at 11 PM Pacific time (2 AM Eastern time May 10) we’ll be performing brief scheduled maintenance on our mail servers. (We’ll be adding more RAM and adding more disk space to make sure that our mail servers continue to keep up with the growth in our service.) This requires restarting, which takes about five minutes, so you will see a brief period of about five minutes where you are unable to connect to our mail servers. No mail will be lost, of course; it will be queued and available after the maintenance.
We apologize for the inconvenience this causes. We schedule this kind of maintenance for late Saturday night/early Sunday morning (the least busy time of the week) to minimize the impact.
The “farnsworth” Web server locked up and needed restarting again last night at about 9:04 PM Pacific time, causing another short outage for some customers. (A similar problem happened Monday night.) To make sure this doesn’t happen again, we’ll be replacing the entire server (switching it with a spare server) at about 11 PM (Pacific) tonight, which will result in about 5 minutes of downtime.
We’re also taking this opportunity to upgrade the hardware on one of our mail servers to allow for future growth; customers (even those with accounts on other Web servers besides the farnsworth server) may see a short (approximately 5 minute) interruption in their ability to retrieve e-mail between 11 PM and midnight.
We apologize for any inconvenience this causes — as always, we’re committed to the highest possible levels of reliability.
