The case of the dropped valve

Part failure while under full load

Authored by: Martin Leduc, Sept 2008

Brought to you by www.dieselduck.net, comments to webmaster@dieselduck.net

Engineering watch keeping is all about setting routines. When you have a predictable operation you have predictable results. Sometimes on watch, machines just seem to develop a mind of their own and wreck your routine, this is such a story.

rh de1 failure.01_th.jpg 3.0K
RH DE1 Failure.01.jpg

rh-de1-failure.02_th.jpg 3.8K
RH-DE1-Failure.02.jpg

rh-de1-failure.03_th.jpg 3.0K
RH-DE1-Failure.03.jpg

rh-de1-failure.04_th.jpg 4.0K
RH-DE1-Failure.04.jpg

I was on the 8-12 watch one fine evening in June 2006, all was routine, a few alarms here and there, nothing really out of the ordinary, which is a good thing. In this career, you have to have a sense of adventure, this preferably involves shore leave, not an engine failing under full load. The ship I was working on at the time, was the Rhapsody of the Seas. A 78,491 gross tons passenger ship launched in 1997, by the French yard, Chantier de l'Atlantic, at a cost of $275 million dollars. Power for this vessel is derived from a diesel electric system, consisting of four Wartsila engines driving Alstom alternators; ship's propulsion is achieve by two 17 MW Cegelec electric motors to give the ship a top speed of just under 24 knots.

The engines are Wartsila Vasa 12V46C; four stroke, turbo charged, 12 cylinder engine in a vee configuration turning at a steady 514 rpm. They produce 12,600kW (16,900 hp), burning heavy residual fuel (380cst), and are usually run at 85-90% load when under way. Royal Caribbean International is quite well versed in this engine, having around 150 copies of this engine operating in their fleet. They require a constant stream of attention but are overall reliable and simple in their design.

It was a quiet night, most crew not on duty at the time were on the aft mooring deck enjoying a BBQ put on by the engine department. The Croatians were manning the BBQ, and delighted the crew with cooking skills from a home far away, displayed on the freshly caught fish, obtained at our last stop in the weekly cruise, Cozumel. My motorman was doing his rounds, with nothing out of the ordinary to report. Then out of the blue engine 1 went into alarm. Below is the report I wrote for the chief, which pretty much gives all the details from my perspective...

Date: June 26th 2006
TO: Chief Engineer
FROM: Martin Leduc, 2nd Engineer
RE: Statement of Events surrounding Diesel Engine 1 failure on Rhapsody of the Seas, June 21st 2006

At 21:31:21 on the evening of Wednesday June 21st 2006, “Exhaust Valve 1 Hi Hi Temperature” deviation alarm occurred on cylinder 6A of Diesel Engine 1 (DE1). Shortly after, at 21:31:27 a “Low HT FW Inlet” alarm and two seconds later “Exhaust Gas Low Deviation” alarms occurred on both exhaust valves of cylinder1A.

These were the first indication that anything was amiss with DE1; up to that point the watch was quiet with routine alarms occurring sporadically. At the time, the ship was at sea and in transit with DE1, 2 and 4 online, loaded to approximately 85% capacity. In the following 11 seconds, over 30 alarms were recorder, mostly temp dev, at which point the control system activated “Step 1 Shutdown” procedure, automatically, at 21:31:40.

In the mean time, a page to the Chief Engineer Jr. requesting assistance was sent, and engine parameters were being reviewed using the ECR’s NorControl workstations to ascertain the source of the issues. It quickly became evident that the engine was suffering a major mechanical failure and at 21:32:07 the engine was stopped using the “Emergency Stop” button from the ECR. The circuit breaker immediately trip and the engine shut down; the propulsion system automatically adjusted to the lost of available power. Bridge was notified of the situation.

The Chief Engineer and the First engineer were paged, and the duty motorman dispatched to access local situation of the engine, as no abnormalities were visible on CCTV system. By that time, Chief Jr was in the ECR and proceeded to the engine to survey. No visible damage from the exterior was evident, but water was continuously coming from charge air cooler drains. The Chief Engineer and First Engineer attended the ECR at this time. Shortly after the Captain was notified by the C/E of the situation, and the engine mechanic day team was summoned as well.

In all, the alarm printer logged over 175 alarm points regarding this incident. The following day the cylinder head on 1A was removed and serious damage was observed. All four valves were broken from their stem and had come loose within the cylinder damaging the exhaust seats; piston crown, liner and the A bank turbocharger.

Martin Leduc
2 Engineer
Rhapsody of the Seas
print-out-sm_th.jpg 2.8K
June 25th 2006

As you can ascertain from the timeline, things happen fast. Here you can see a complete list of alarms that activated during this incident. The propulsion system on this series of ship, is very robust, so pushing the emergency stop button, which immediately disconnects the circuit breaker and activates fuel shut off valves at the engine, does not cause any surprise over speeds or other damaging conditions to the rest of the plant. I think this is a critical part of a response while on watch, to contain the incident at the earliest opportunity, not many other ship did I feel confident on doing such a manoeuvre in such short decision time. 

As stated in my report above, the next day the cylinder head was removed from the engine, and it was pretty obvious that a valve had dropped. Below are some of the pictures of the engine crew pulling the piston out. As you can see the piston crown was heavily damaged from the exhaust and intake valves, and their seats coming apart. The piston was more or less seized in the liner and, as you can see in the pictures, the lifting lugs in the crown were inaccessible due to damage, so a steel bracket was welded to the crown. With the liner bracket in place, both piston and liner were lifted out of the engine. 

rh-de1-failure.08_th.jpg 3.9K
RH-DE1-Failure.08.jpg

rh-de1-failure.09_th.jpg 4.2K
RH-DE1-Failure.09.jpg

rh-de1-failure.06_th.jpg 4.4K
RH-DE1-Failure.06.jpg

rh-de1-failure.07_th.jpg 4.6K
RH-DE1-Failure.07.jpg

rh-de1-failure.10_th.jpg 4.2K
RH-DE1-Failure.10.jpg

rh-de1-failure.11_th.jpg 3.9K
RH-DE1-Failure.11.jpg

rh-de1-failure.12_th.jpg 4.6K
RH-DE1-Failure.12.jpg

rh-de1-failure.13_th.jpg 3.1K
RH-DE1-Failure.13.jpg

Above you can see one of the valves firmly imbedded into the crown of the piston; pretty amazing really. Below are the pieces recovered from cylinder. You can see a considerable amount debris was recovered. The valve seats of all four valves are somewhat visible along with the hammered valves. 

rh-de1-failure.16_th.jpg 5.2K
RH-DE1-Failure.16.jpg
rh-de1-failure.17_th.jpg 4.3K
RH-DE1-Failure.17.jpg

rh-de1-failure.14_th.jpg 3.8K
RH-DE1-Failure.14.jpg

rh-de1-failure.15_th.jpg 4.5K
RH-DE1-Failure.15.jpg

It became pretty obvious that this engine was going to need a major overhaul, even though it was premature. Although we have a considerable amount of personnel assigned to the engine room, such a large time consuming task of rebuilding the entire engine would tax our capabilities of smooth operation. The chief engineer shuffled his priorities and requested that the company's engine riding team join the ship to undertake this rebuild.

Royal Caribbean has several engine riding team that roam the fleet. A half dozen guys, specializing in rebuilding the Wartsila 46. They fly in with their specialized tools and some parts and take the lead in tearing down and rebuilding the engine with the assistance of the engine room day team, while the ship continues to sail. Once they are done, they go to another ship and start over. In this case, they were schedule to rebuild engine 3, so they just carried on with that after engine 1.

The amount of debris resulting from failure in cylinder one necessitated that all the heads be removed and inspected for debris, the exhaust and intake manifold be inspected and cleaned. Both turbo chargers suffer damage from the debris as well; "A bank" turbo charger from pieces going through it, then coming back down the uptake into the "B bank" turbo charger - what a mess.  Below is Guzman, he was the First Engineers, in charge of that particular riding team, next to him is Lali, his relief in training. From the pictures, I am sure you can get a grasp of the logistic of such a rebuild, while maintaining a regular sailing schedule - indeed amazing.   

rh-de1-failure.05_th.jpg 3.7K
RH-DE1-Failure.05.jpg

rh-de1-failure.18_th.jpg 4.1K
RH-DE1-Failure.18.jpg

rh-de1-failure.19_th.jpg 4.3K
RH-DE1-Failure.19.jpg

rh-de1-failure.20_th.jpg 3.1K
RH-DE1-Failure.20.jpg

 

rh-de1-failure.21_th.jpg 3.9K
RH-DE1-Failure.21.jpg

rh-de1-failure.22_th.jpg 4.1K
RH-DE1-Failure.22.jpg

 

rh-de1-failure.23_th.jpg 4.5K
RH-DE1-Failure.23.jpg

rh-de1-failure.24_th.jpg 4.1K
RH-DE1-Failure.24.jpg

rh-de1-failure.25_th.jpg 4.7K
RH-DE1-Failure.25.jpg

rh-de1-failure.26_th.jpg 4.0K
RH-DE1-Failure.26.jpg

Guzman theorizes that the valve failed due to corrosion on the stem of the exhaust valve. This corrosion was evident on the other cylinders' valves as well; a failure well premature to their expected life. This corrosion of the stem was most likely precipitated by poor quality fuel. The corrosion weakened the valves structure and eventually it broke, the pieces damaging the other valves and valve seats causing the rubble seen in the pictures above. The smaller pieces were then carried through the turbo charger by gases, severely damaging it. The exhaust seat damage, cause the coolant to flood the cylinder, which explained the turbocharger drain continuous leakage of water, shortly after the onset of the incident.

After a week or so, the engine was fully rebuild and started up without anything out of the ordinary. The Chief Engineer approved the "break in" which follows a very specific set of loading for a set time, over 12 hours or so. But more surprises where in store for us.

Towards the end of the break in period, on the engine final rest before full load, more temperature deviation alarms went off indicating a failure of the combustion cycle within the cylinder. It was determined that the turbocharger was again damaged. 

A considerable amount of head scratching then occurred - did you check the turbo charger before installing? was it the right spare? bad rebuild... ? Well indeed every thing was in its place; the only explanation was to be a bit of stunner! You see Diesel Engine 1 is on the starboard side of the engine room, with an uptake going through the casing located amidships. The exhaust trunking therefore does a horizontal trip across the engine room before going vertical. The theory was, that metal fragments from the valve and the damaged turbo managed to get into that horizontal section of the exhaust trunking, and slowly vibrated back down the turbo after the break in, under full load. What a pisser! Let that be a lesson for the future, check that trunking.

Brought to you by www.dieselduck.net, comments to webmaster@dieselduck.net