Dates: January 6, 7, 8 & January 13, 14, 15

Time: Maintenance window will be open from 8:00pm mountain - 2:00am mountain time

Expected Impact: Brief periods of latency and packet loss during the maintenance windows.

Purpose of Work: We will be performing various network upgrades, including:

[1] Bringing optical amplifiers online to increase the capacity on our dark fiber ring

[2] Adding new line cards to our MX240 routers and grooming existing circuits over to them

[3] Replacing our aging MX80s and EX4500 switches at 1801 California St with Juniper QFX5100 switches

During the maintenance window, we will be providing updates regarding specific activities that will be taking place during these windows.  Unfortunately, due to the scope of the work we need to perform, and the interrelatedness and dependencies of all of this work, we are bulk scheduling maintenance windows.

2017-01-10 23:36 MST:  During our network maintenance for the evening, we were operating under reduced redundancy.  Unfortunately, as our luck would have it, one of our border routers decided to coredump, causing a network wide outage for approximately 25 minutes.  During these 25 minutes, we were rolling back our previous changes made to effect the maintenance window, which involved reverting both physical and logical networking configurations.  Connectivity was disrupted at 22:50 and restored at 23:15.  We are still actively working to determine more about the outage.  The maintenance window may be extended to address any issues that remain related to this unexpected problem.

2017-01-11 00:43 MST:  All systems are working as expected at this time.  All further maintenance is canceled, until we can work with our vendor to get to the bottom of the issues that occured this evening.

Friday, January 6, 2017

« Back