Navigatoris is aware of a network issue impacting our network in Los Angeles (HQ), this is widespread effect affecting our upstream Internet Service Provider Xeex Communications (http://xeex.com).
This is impacting all of XEEX's customers including Navigator Internet Solutions, we are awaiting for resolution from XEEX and have contacted them regarding this.
Navigatoris fiber internet provider has suffered a fibre cut in our facility, the following Reason For Outage (RFO) has been provided to us.
* Timeline of the outage - 6:03AM PST - 7:54AM PST
* Root cause of failure - Fiber cut to 10G link between
BBR01.LAX02 and BBR02.LAX02
* Description of what has happened - At approximately 6:03AM PST
our monitoring system alerted our NOC to a loss of connectivity between
two of the backbone routers in the LAX02 facility at 600 West 7th
Street, affecting all single-homed customers on the 3rd floor. After
investigating possible optic, line card, and other active failures Xeex
engineers determined the fault to be somewhere along the fiber path
connecting the two routers. After performing a number of loopback tests
and analyzing the light readings, engineers narrowed down the fiber cut
to somewhere in the conduit running between the 3rd and 6th floors of
the building. Immediately thereafter, engineers began the process of
re-running the fiber span up to the 3rd floor conduit closet. At
approximately 7:54AM PST the re-built fiber span was brought online and
connectivity was re-established.
* Specific action items to work on to prevent this from happening
again - We are currently attempting to discover what caused the fiber
cut within the conduit to better understand how to prevent this from
happening again. At the same time, we are encouraging customers to
multi-home their connections to the Xeex network so that they can take
advantage of the diverse physical paths from the third floor.
We sincerely apologize for any inconvenience this outage may have
caused. Rest assured we are working diligently to ensure that this type
of service interruption is mitigated in the future.
If there are any further questions or concerns that you have, please
feel free to open up a ticket in our portal system or send an e-mail to
our NOC at noc @ xeex.com.
Thank you very much.
Thank you for your cooperation.
Thursday, April 22, 2010