Unscheduled network outage (resolved)

Between 9:52 and 11:06 PM Pacific time on January 10, a complete network failure at our primary data center caused an unscheduled outage that resulted in all services (all Web sites and e-mail) being unreachable from the Internet.

This problem has been resolved and all services are now available. We are waiting for a full report from the data center personnel so that we can determine the cause and ensure that it won’t recur.

We sincerely apologize to our customers who were affected by this. This kind of outage is not normal (it’s the longest outage we’ve experienced in more than four years), and we know it’s not acceptable to our customers who rely on our services. We’ll post a followup message with more details when they become available.

Update Friday 10 AM: As a clarification, we should also have originally mentioned that no e-mail is lost during this kind of outage: it’s delivered after the issue is resolved. While some messages were certainly delayed, they were all properly delivered afterward.