Break reports

Here are listed all breaks in HIIT's IT services.

Break in file server fs.hiit.fi at 2013-02-14 16:58 - 20:36

Schedule: 
2013-02-14 16:58 to 20:36
Duration: 
3h 38min
Affected services: 
fs.hiit.fi, shell.hiit.fi, hosted static web pages
Description: 

HIIT's file server fs.hiit.fi first partially stopped responding to SMB requests and soon after that also NFS traffic slowed down. Due to that and longish uptime the server was rebooted.

Pending operating system and firmware updates were also planned to be installed.

Update at 20:56: File service and all services depending on it are up and running. Firmware updates were skipped due to server being somehow in a weird state; firmware update hung (quite likely the first time ever). To reset the server it was removed and reinserted to the blade enclosure. Hopefully this fixes slow performance in remote management (iLO2).

Break in HPC IPv6 network connections at 2012-12-18 14:50-15:14

Schedule: 
2012-12-18 14:50 to 15:14
Duration: 
24 min
Affected services: 
University of Helsinki department of computer science's Ukko cluster, HIIT-HPC
Description: 

HPC network was moved to another router. Due to non-activated IPv6 routing IPv6 traffic wasn't routed neither in or out from HPC-VLAN.

Configuration has now been corrected to prevent this kind of break from happening again.

Break in IPv6 network connections at 2012-12-17 13:49-13:52

Schedule: 
2012-12-17 13:49 to 13:52
Duration: 
3 min 2 sec
Affected services: 
IPv6 traffic
Description: 

Due route- network rearrangements, IPv6 routes were inconsistent and IPv6 traffic didn't flow to and from HIIT's production network.

Break in network connections at 2012-12-11 18:33-18:36

Schedule: 
2012-12-11 18:33 to 18:36
Duration: 
3 min 2 sec
Affected services: 
Ukko- cluster, shell.hiit.fi, virtualised services (HIIT and UH/CS)
Description: 

Router firmware upgrade stopped traffic for approximately 3 minutes.

Break in shell.hiit.fi 2012-11-20 08:00 -09:00

Schedule: 
2012-11-20 08:00 to 09:00
Duration: 
1 h
Affected services: 
Entry and general purpose server shell.hiit.fi
Description: 

Shell.hiit.fi will be moved to new virtual infrastructure. Operation takes one hour and during that time shell  won't be accessible. Users logged in are forced to logout when the operation starts.

Update at 8:21: The Break is over.

 

Pages