We had a couple of brief network interruptions today (at about 8:50am and 12:49pm Pacific time). We are investigating them, and will update this post with more details later.
Updated 2012-07-19 2:23PM Pacific: One of our upstream network providers has traced this to what appears to be some very brief, very high-packet-per-second attacks. The attacks have not recurred, and we are continuing to monitor all systems.
Between 10:00 PM and 11:00 PM Pacific time on Friday March 9, 2012, we’ll be updating the MySQL database software on all our hosting servers. This will cause a Web site service interruption of about 30 seconds for some customers at some time during this period. E-mail will not be affected.
This maintenance is necessary to install a mandatory MySQL security update that will upgrade the MySQL version to 5.1.61. We apologize for any inconvenience this causes.
Update 10:13 PM: The maintenance was completed with less than 30 seconds downtime on each server. Customers should not notice any changes, but as always, don’t hesitate to contact us with any questions or problems.
On Saturday, February 18, 2012 between 10:00 and 11:00 PM Pacific time, we’ll be upgrading the Apache Web server software on each of our Web servers.
Most customers will not notice anything, but the upgrade will cause approximately 30 seconds of slow Web page loading at some point during that hour as we delay incoming connections at the network level.
This maintenance is necessary to apply security and reliability fixes released by the Apache developers. (We’ve been using the upgraded version on our Webmail servers for several days, so it’s well tested.)
Update: The maintenance was completed at 10:03 PM Pacific time.
Between 5:34 PM and 6:10 PM Pacific time December 12, many customers experienced a complete outage of their sites (and of our own www.tigertech.net and mail.tigertech.net sites).
This was caused by the failure of a hardware Ethernet switch in one of our server cabinets, cutting off all access to the servers that plug into it. The Ethernet switch began working after being physically unplugged and plugged in again, but since we do not know why it failed, it will be completely replaced tonight as a result of this incident.
This is the same model of Ethernet switch that we’ve been using in all our cabinets for years, so we don’t believe it is a general problem with the hardware in question.
We sincerely apologize for this incident. We take reliability seriously, and we don’t consider it acceptable.
Update 1:20 AM: The failed Ethernet switch was replaced with no further downtime.
Our previously-announced move to the new data center is continuing. The old data center was hit with a lot of network attacks throughout the day today. We are very glad to be moving to the new center!
E-mail should be fully working for everyone. Many customers have been moved to the new data center, and we’ll be continuing to move the remaining sites as quickly as possible (while of course being very careful about data integrity and site uptime). Sites which have not yet been moved are still working at the old data center. The old data center has been up for several hours now with only very occasional interruptions (due to attacks). They might still experience occasional bouts of downtime if the attacks continue.
If you want to know if your Web site has been moved to a server at the new data center, please see the update at the bottom of this page.
Between 10:32 AM and 10:47 AM Pacific time this morning (October 3), our monitoring systems detected high “packet loss” from one “network backbone”, which may have caused slow connections or timeouts for some customers. The monitoring systems show that this issue is resolved.
The data center that experienced network problems earlier today has just informed us that they’ll be performing emergency maintenance on all their network routers tonight (Thursday, September 29, 2011) between 6:00 and 7:00 PM Pacific time.
During that hour, there may be up to five minutes total of network connectivity problems that makes some sites load slowly or fail to load.
In an apparent continuation of last night’s incident, many sites we host were intermittently unavailable between 12:01 PM and 1:20 PM Pacific time today (September 29, 2011). This also caused slow mail delivery and reduced spam filtering effectiveness until around 2:00 PM (no mail was lost, of course).
All systems are operating normally as of 2:15 PM.
A problem at our old data center (the one we’re moving sites from this month) caused some sites to be intermittently unavailable between 10:22 and 10:47 PM Pacific time.
Over the next four weeks, we’ll be migrating customer Web sites to upgraded servers. The servers have updated software (and upgraded hardware in some cases), and are also located in a data center with increased power reliability.
For most customers, these changes will be completely unnoticeable. However, a very small number of customers might notice software differences or experience up to five minutes total of “downtime” at some point. We recommend reading through this entire post for details.
