As a number of you seemed to be interested in the Domino Effect I thought I might share an experience I had only last year. I have taken the original report and hacked it about to preserve the innocent and make it a bit less technical and a bit more readable. I have also about halved the content so you can imagine what the original read like!!
The ship was alongside with Diesel Generators Two and Three providing power.
At 12.25 PM the ship experienced a failure of one of the Uninterruptible Power Supply (UPS) Units. This particular UPS supplies a lot of the monitoring and control equipment in the Control Room itself, equipment that must always remain powered up.
Unfortunately, however, the unit did not completely fail and automatically change over to normal power supply but it failed in such a way that the voltage output from the unit was fluctuating from it’s normal voltage, thus causing the shutdown of various systems within the Control Room. Number Two Generator breaker tripped off, as it's power management system is supplied from this UPS, and the remaining Generator, Number Three, although its breaker remained closed supplied power with the frequency varying alarmingly.
Because the UPS did not shut down completely it was not immediately obvious what the cause for the loss of systems in the Control Room was even when the Unit was checked. When it was realised that this was the cause of the loss of control the UPS was changed over manually to normal power supply and the systems in the Control Room returned to normal. Number Two Diesel Generator breaker was closed again and Diesel Generator Number Five was also started and put on line at 12.44. At approximately 13.00 hrs the systems within the Control Room were returned to normal operation.
Due to Number Two Generator breaker tripping the sudden load on Number Three Generator combined with the varying frequency generated by that machine caused the shut down of all the main air conditioning plant. When the systems in the Control Room were stabilised the Chillers were restarted. It was noted that the Chilled Water system, which distributes the cooling effect of the chillers up to the fan rooms, was unstable with the pressure fluctuating. The expansion tank went into low-level alarm and considerable amounts of air were bled from the system in the engine room. During approximately the next hour many attempts to restart chiller machines were attempted and continuous bleeding off of air in the system was continued. The Chilled Water Expansion tank feed was changed over to stand by and the system fed directly from the technical water system however it became apparent that the Chilled Water system was not stable and it was suspected that a leak in a Fan room was the cause. Chiller machines had been intermittently running during this time but due to fluctuating Chilled Water pressure had been tripping off.
All available personnel were dispatched to search all fan rooms for possible leaks. It was then that a broken Chilled Water main line of 4 inch diameter was discovered in a Fan Room. It is believed that this pipe failed when the chilled water pumps were all being restarted simultaneously. Engineers were unable to stem the flow of water from this pipe (have you ever stood in front of a fire hose?) so the sections of the system that it fed were isolated. The leak had caused extensive flooding and had covered a large amount of electrical equipment within the fan room. At this point three chiller machines were started however they once again remained unstable due to the lack of flow around the system caused by the isolated sections (someone upstairs was really having a laugh!). These machines therefore also tripped off as a result of high differential pressures across the Chiller Evaporators.
Repairs were affected on the burst pipe in the Fan Room however it was then discovered that the several tons of water draining from this Fan Room was too much for the drain system to cope with and thus draining water backed up into another Fan Room on a lower deck. This flooded the lower Fan Room to a level above the lower level of the electrical supply panel in that room necessitating the isolation of that electrical panel. This then meant that Fans were lost for the main propulsion systems so propulsion electrical Filters were isolated to maintain temperature. At this point then Main Propulsion was no longer available and the bridge was informed and I checked my CV was up to date.
When repairs were completed to the burst pipe in the upper fan Room the isolated system was reflooded and, although great care was taken to flood the pipework slowly, once again Chiller Machines tripped off due to fluctuating Chilled Water pressures. This point is now approximately 16.00 hrs and I looked like I had been for a swim , which wasn't far from the truth.
The remaining issues were the flooded electrical panel in the lower Fan room and the water that had entered the panels and machinery in the upper fan Room. Both these areas were worked on simultaneously as propulsion remained unavailable. The electrical panel in the lower Fan Room was put back on line at 17.00 hrs, after drying and insulation readings returned to acceptable levels, and propulsion services reinstated to allow departure. Over the period of the next two and a half hours all fans and electrical equipment was reinstated after drying and cleaning of various starter units.
We sailed 10 minutes late and hardly anyone realised we had had no air conditioning for about 4 hours.
The ship was alongside with Diesel Generators Two and Three providing power.
At 12.25 PM the ship experienced a failure of one of the Uninterruptible Power Supply (UPS) Units. This particular UPS supplies a lot of the monitoring and control equipment in the Control Room itself, equipment that must always remain powered up.
Unfortunately, however, the unit did not completely fail and automatically change over to normal power supply but it failed in such a way that the voltage output from the unit was fluctuating from it’s normal voltage, thus causing the shutdown of various systems within the Control Room. Number Two Generator breaker tripped off, as it's power management system is supplied from this UPS, and the remaining Generator, Number Three, although its breaker remained closed supplied power with the frequency varying alarmingly.
Because the UPS did not shut down completely it was not immediately obvious what the cause for the loss of systems in the Control Room was even when the Unit was checked. When it was realised that this was the cause of the loss of control the UPS was changed over manually to normal power supply and the systems in the Control Room returned to normal. Number Two Diesel Generator breaker was closed again and Diesel Generator Number Five was also started and put on line at 12.44. At approximately 13.00 hrs the systems within the Control Room were returned to normal operation.
Due to Number Two Generator breaker tripping the sudden load on Number Three Generator combined with the varying frequency generated by that machine caused the shut down of all the main air conditioning plant. When the systems in the Control Room were stabilised the Chillers were restarted. It was noted that the Chilled Water system, which distributes the cooling effect of the chillers up to the fan rooms, was unstable with the pressure fluctuating. The expansion tank went into low-level alarm and considerable amounts of air were bled from the system in the engine room. During approximately the next hour many attempts to restart chiller machines were attempted and continuous bleeding off of air in the system was continued. The Chilled Water Expansion tank feed was changed over to stand by and the system fed directly from the technical water system however it became apparent that the Chilled Water system was not stable and it was suspected that a leak in a Fan room was the cause. Chiller machines had been intermittently running during this time but due to fluctuating Chilled Water pressure had been tripping off.
All available personnel were dispatched to search all fan rooms for possible leaks. It was then that a broken Chilled Water main line of 4 inch diameter was discovered in a Fan Room. It is believed that this pipe failed when the chilled water pumps were all being restarted simultaneously. Engineers were unable to stem the flow of water from this pipe (have you ever stood in front of a fire hose?) so the sections of the system that it fed were isolated. The leak had caused extensive flooding and had covered a large amount of electrical equipment within the fan room. At this point three chiller machines were started however they once again remained unstable due to the lack of flow around the system caused by the isolated sections (someone upstairs was really having a laugh!). These machines therefore also tripped off as a result of high differential pressures across the Chiller Evaporators.
Repairs were affected on the burst pipe in the Fan Room however it was then discovered that the several tons of water draining from this Fan Room was too much for the drain system to cope with and thus draining water backed up into another Fan Room on a lower deck. This flooded the lower Fan Room to a level above the lower level of the electrical supply panel in that room necessitating the isolation of that electrical panel. This then meant that Fans were lost for the main propulsion systems so propulsion electrical Filters were isolated to maintain temperature. At this point then Main Propulsion was no longer available and the bridge was informed and I checked my CV was up to date.
When repairs were completed to the burst pipe in the upper fan Room the isolated system was reflooded and, although great care was taken to flood the pipework slowly, once again Chiller Machines tripped off due to fluctuating Chilled Water pressures. This point is now approximately 16.00 hrs and I looked like I had been for a swim , which wasn't far from the truth.
The remaining issues were the flooded electrical panel in the lower Fan room and the water that had entered the panels and machinery in the upper fan Room. Both these areas were worked on simultaneously as propulsion remained unavailable. The electrical panel in the lower Fan Room was put back on line at 17.00 hrs, after drying and insulation readings returned to acceptable levels, and propulsion services reinstated to allow departure. Over the period of the next two and a half hours all fans and electrical equipment was reinstated after drying and cleaning of various starter units.
We sailed 10 minutes late and hardly anyone realised we had had no air conditioning for about 4 hours.
Comment