Scheduled maintenance March 9 & 10, 2018 for some servers (completed)

Update 11:55 PM Pacific time: The maintenance described below was completed successfully and all services are running normally.

Over the last year, we’ve been slowly upgrading our servers from Debian Linux version 7 (codename “wheezy”) to version 8 (codename “jessie”). We’ll be finishing that process over the next few weeks, with brief scheduled maintenance on each server.

The upgrade requires that each hosting server be taken offline for a few minutes over a three hour maintenance window and then restarted, causing brief scheduled “downtime” for websites and email on that server. The total downtime for any server should not exceed ten minutes during this three hour period.

Mail arriving while a server is being restarted will be queued and delivered after a short delay. No mail will be lost.

This coming weekend, we’ll be updating some (not all) web servers:

  • Friday, March 9, 9:00-11:59 PM Pacific: servers ending in digit “2” (completed)
  • Saturday, March 10, 9:00-11:59 PM Pacific: servers ending in digit “1” (completed)

So, for example, the “web12” server will be updated on March 9, and the “web01” server will be updated on March 10. This page explains how to find which server a site is on.

(Servers ending with other digits have already been updated in maintenance in previous weeks. This will complete the upgrades.)

Although the final step of the upgrade modifies hundreds of software packages on the server, we have tested it extensively don’t expect most customers to notice any change. Your website and email should continue working as they always have. However, if you do have any trouble, don’t hesitate to contact us.

Scheduled maintenance March 2 & 3, 2018 for some servers (completed)

Update 11:18 PM Pacific time: The maintenance described below was completed successfully and all services are running normally.

Over the last year, we’ve been slowly upgrading our servers from Debian Linux version 7 (codename “wheezy”) to version 8 (codename “jessie”). We’ll be finishing that process over the next few weeks, with brief scheduled maintenance on each server.

The upgrade requires that each hosting server be taken offline for a few minutes over a three hour maintenance window and then restarted, causing brief scheduled “downtime” for websites and email on that server. The total downtime for any server should not exceed ten minutes during this three hour period.

Mail arriving while a server is being restarted will be queued and delivered after a short delay. No mail will be lost.

This coming weekend, we’ll be updating some (not all) web servers:

  • Friday, March 2, 9:00-11:59 PM Pacific: servers ending in digits “6” and “5” (completed)
  • Saturday, March 3, 9:00-11:59 PM Pacific: servers ending in digits “4” and “3” (completed)

So, for example, the “web06” server will be updated on March 2, and the “web13” server will be updated on March 4. This page explains how to find which server a site is on.

(Servers ending in “0”, “9”, “8” and “7” have already been updated. Servers ending in “2” and “1” will be updated the following weekend; we’ll post a separate announcement about that.)

Although the final step of the upgrade modifies hundreds of software packages on the server, we have tested it extensively don’t expect most customers to notice any change. Your website and email should continue working as they always have. However, if you do have any trouble, don’t hesitate to contact us.

Scheduled maintenance February 23 & 24, 2018 for some servers (completed)

Update 11:18 PM Pacific time: The maintenance described below was completed successfully.

Over the last year, we’ve been slowly upgrading our servers from Debian Linux version 7 (codename “wheezy”) to version 8 (codename “jessie”). We’ll be finishing that process over the next few weeks, with brief scheduled maintenance on each server.

The upgrade requires that each hosting server be taken offline for a few minutes over a three hour maintenance window and then restarted, causing brief scheduled “downtime” for websites and email on that server. The total downtime for any server should not exceed ten minutes during this three hour period.

Mail arriving while a server is being restarted will be queued and delivered after a short delay. No mail will be lost.

This coming weekend, we’ll be updating some (not all) web servers:

  • Friday, February 23, 9:00-11:59 PM Pacific: servers ending in digits “0” and “9” (completed)
  • Saturday, February 24, 9:00-11:59 PM Pacific: servers ending in digits “8” and “7” (completed)

So, for example, the “web10” server will be updated on February 23, and the “web07” server will be updated on February 24. This page explains how to find which server a site is on.

(Additional servers will be updated the following weekends; we’ll post separate announcements about that.)

Although the final step of the upgrade modifies hundreds of software packages on the server, we have tested it extensively don’t expect most customers to notice any change. Your website and email should continue working as they always have. However, if you do have any trouble, don’t hesitate to contact us.

Brief outage on web07 server Feb 7 2018 (resolved)

Between 04:23 AM Pacific time (February 7 2018) and 04:33 AM Pacific time (February 7 2018), websites on our “web07” server were inaccessible while we restarted the server due to an unexpected kernel software issue.

All services are now running normally, and we are investigating the initial cause of the problem. We apologize for the inconvenience this caused.

Mailman mailing list software upgraded to version 2.1.26

The authors of the Mailman mailing list software we provide for customers have recently released version 2.1.26 to fix several bugs.

We’ve upgraded the Mailman software on our servers as a result.

Users of Mailman lists shouldn’t notice any changes, but as always, don’t hesitate to contact us if you have any questions or see any problems.

Brief scheduled maintenance January 8, 2018 (completed)

Between 9:00 PM and 11:59 PM Pacific time on Monday, January 8, 2018, each of our hosting servers will be restarted. This will cause a brief interruption of service (less than 5 minutes) for each site at some point during this period.

Read the rest of this entry »

Web02 RAID and MySQL problems (resolved)

Between 05:46 AM Pacific time and 06:58 AM Pacific time (December 29 2017) some Web sites on our web02 server were inaccessible. This was caused by the failure of a disk in the RAID array. When the disk failed MySQL continued to read corrupted data from it’s file cache rather than using another disk in the RAID array. The cache has now been cleared and the server restarted. This was unrelated to the problems on our web13 server earlier in the day.

Web13 MySQL database problems (resolved)

Between 6:06 PM Pacific time (December 28 2017) and 05:15 AM Pacific time (December 29 2017) some Web sites on our web13 server were inaccessible for periods of time due to MySQL database corruption problems. During that time customers will likely have experienced a “Error Establishing Database Connection” message when viewing their Web site.

By 8:49 PM Pacific time (December 28 2017) most Web sites were working although a small number needed databases manually restoring by our technicians. That process was completed by 05:15 AM Pacific time (December 29 2017).

We don’t consider this acceptable and will strive to make sure this doesn’t recur. Finally we apologize for the inconvenience this caused.

Brief scheduled maintenance on web07 server December 23, 2017 (completed)

At approximately 9:30 PM Pacific time on Saturday, December 23 2017, the web07 server will be restarted.

This will cause a brief interruption of service (less than 3 minutes) for sites and email on this server. Sites on other servers will not be affected.

Read the rest of this entry »

SSLv3 disabled on our webmail servers

We’ve updated the SSL/TLS security settings on our webmail servers to match current “best practices” for security, disabling “SSLv3”.

Our customers shouldn’t notice any changes. (We made this change on our main website some time ago with no reports of problems, and many of the largest sites on the Internet have done the same.) We’re just mentioning this so that people know to contact us in the unlikely event they do have any trouble.

That said, if you do have any trouble, it’s probably because you’re using an outdated, insecure web browser that you should update. You can check your browser by visiting www.howsmyssl.com. If you can’t update it, using a different browser on your computer will probably help.