Denial of service attack (resolved)

Beginning at 2:16 AM Pacific time this morning, we began experiencing a “distributed denial of service” attack aimed at our “flexo” Web server.

The attack used more than 2 Gbps of network bandwidth from several thousand different IP addresses. This is an extremely high amount of traffic, saturating even our network connections.

The problem caused most of our servers to become unreachable (or very slow) from the Internet.

We restored service to all servers except the flexo Web server at 2:59 AM (by getting our network providers to block all packets for certain IP addresses). We restored service to the flexo server at 3:29 AM (by getting them to identify and block specific characteristics of the attack).

All services are now operating normally, and all delayed incoming mail has been delivered.

We take reliability seriously. Unfortunately, this is by far the largest attack we’ve seen on our network in ten years. We sincerely regret and apologize for the impact this had on our customers.