Our business offices will be closed on Monday, December 26 to observe the US legal holiday. As always, our support staff will be providing same-day support for time-sensitive issues via our ticket and e-mail systems. However, questions that aren’t time-sensitive (including most billing matters) may not be answered until Tuesday, and telephone support (via callbacks) will be available only for urgent problems.
Update: The problems described below were resolved by Comcast around 11:00 AM Pacific time and have not recurred since. We’re cautiously marking this issue closed, but continuing to monitor it.
We’ve received scattered reports of high “packet loss” to a few Comcast locations (but not most). Packet loss can cause pages to load slowly in some cases.
Between 5:34 PM and 6:10 PM Pacific time December 12, many customers experienced a complete outage of their sites (and of our own www.tigertech.net and mail.tigertech.net sites).
This was caused by the failure of a hardware Ethernet switch in one of our server cabinets, cutting off all access to the servers that plug into it. The Ethernet switch began working after being physically unplugged and plugged in again, but since we do not know why it failed, it will be completely replaced tonight as a result of this incident.
This is the same model of Ethernet switch that we’ve been using in all our cabinets for years, so we don’t believe it is a general problem with the hardware in question.
We sincerely apologize for this incident. We take reliability seriously, and we don’t consider it acceptable.
Update 1:20 AM: The failed Ethernet switch was replaced with no further downtime.
Our business offices will be closed on Thursday, November 24 to observe the US legal holiday for Thanksgiving.
As always, we’ll provide same-day support for time-sensitive issues via our ticket and e-mail systems. However, questions that aren’t time-sensitive (including most billing matters) may not be answered until Friday, and telephone support (via callbacks) will be available only for urgent problems.
At approximately 8:00 PM Pacific time on November 12, 2011, the “web06” Web server will be restarted.
As a result, for customers on the “web06” server (only), Web site service and the ability to read incoming e-mail will be unavailable for approximately five minutes. Customers on other servers will not be affected.
We’ve been notified by an upstream network provider that they will be performing router firmware upgrades on Saturday, November 5, 2011 between 5:00 and 6:00 PM Pacific time. In the worst case, some customers may see a network interruption of up to five minutes during this period.
Please accept our apologies for any interruption this causes; we’re told that the upgrades are necessary to prevent possible network problems.
(This is the rescheduled time for an original announcement that was postponed.)
Update: The maintenance was completed with approximately three minutes downtime for affected connections.
We’ve been notified that the maintenance previously scheduled for tonight (November 3, 2011) has been canceled and will be rescheduled for a future time.
As a followup to our previous posts about the move to a new primary data center, we want to confirm to our customers that the change was successfully completed.
Due to unrelated network outages at the old data center, we accelerated the original schedule mentioned in that post. Almost all customer sites were moved by October 7, and the remainder (a small handful of customer sites that needed manual intervention due to old software that was incompatible with the Debian Linux software update) were moved as of October 18. Everything is, and has been, working normally.
I want to again take the time to apologize to our customers for the service interruptions that occurred because of the original power problem and the later network problem. They weren’t acceptable. We know you count on us for your success, and we’re constantly working to improve reliability.
Our previously-announced move to the new data center is continuing. The old data center was hit with a lot of network attacks throughout the day today. We are very glad to be moving to the new center!
E-mail should be fully working for everyone. Many customers have been moved to the new data center, and we’ll be continuing to move the remaining sites as quickly as possible (while of course being very careful about data integrity and site uptime). Sites which have not yet been moved are still working at the old data center. The old data center has been up for several hours now with only very occasional interruptions (due to attacks). They might still experience occasional bouts of downtime if the attacks continue.
If you want to know if your Web site has been moved to a server at the new data center, please see the update at the bottom of this page.
We are continuing to experience intermittent problems at Hurricane Electric, the data center that we’ve been using until now. Fortunately, as previously mentioned, we are in the process of moving to a new data center.
Some customers have already been moved to the new data center, and they have not seen any of these problems. We are moving the remaining sites as quickly as possible.
We are extremely upset with the attacks that have been hitting Hurricane Electric, and consider these outages to be unacceptable. It definitely appears that Hurricane Electric is being specifically targeted. You can watch their Twitter feed to see information about the latest attacks and the steps they’re taking to fix the attacks, as well as what they’re doing to try to protect themselves against further ones. While it’s undoubtedly of little comfort, it’s worth noting that these attacks are not directed at our customers or at us, but at Hurricane Electric in general. You can see that other large companies hosted at Hurricane Electric are also being affected, such as Linode.
We apologize for the inconvenience these outages have caused, and hope to have all Web sites moved to the new data center very soon.
