The Fickle Nature of COPVs

As I have already discussed, COPV's are a vast improvement in weight vs. functionality in that quite high pressure COPV's can be manufactured that weigh a small percentage of the heavy wall tank needed to safely contain high pressure gases. When you start adding in extreme temperatures such as LOX (-297 Degrees F) then the metal tanks must also be of high purity stainless steel as well, which gets even more expensive and complicated to design and build.

One of the first groups to work with designing and building COPV's worked at Marshall Space Flight Center in Huntsville. One designer in particular spent much of his career working on developing processes to accurately design and build COPV's rated for very high pressures at very low temperatures. While COPV's have been around for a while the idea of using them inside of cryogen storage tanks was an undertaking that required a lot of research and development. What kind of fibers to use, what kind of helical winding pattern to overlay the tank in layers, what kind of glue to use and how to cure it; all of these factors and many more were relatively unknown in the beginning simply because no one had experience in building these types of tanks.

As luck would have it this designer happened to get some research and development money just about the time that our small testing lab was getting started. When he approached our group about doing failure testing on his COPV's we were able to give him some pretty low bids on doing this testing so it was a marriage of convenience for us both. He needed an inexpensive way to do failure testing on his designs and we needed funds enough to take on the relatively small amount of work this would entail at a low rate.

The first tests we did on these bottles involved hydro testing them to failure. The process involved putting them in a hydro chamber which we would then fill with water to help contain the blast wave. Then, we would fill the test bottle with water and begin pumping it up to its failure pressure. Hydro testing is much preferred to pneumo testing because water is not compressible. Therefore, when the bottle ruptures the pressure is quickly abated with a minimal shock wave as the water escapes the ruptured bottle. When you do pnuemo testing you pressurize the bottle with a gas which is very compressible. As the gas pressure increases until the tank ruptures there is quite a violent release of the stored energy in the gas as it has to completely expand back to its original state.

It is the difference between filling a balloon with air and puncturing it with a pin and filling the same balloon with water and puncturing it with a pin. The balloon filled with air will violently fail as the compressed air escapes the tiny hole and rapidly expands. The balloon filled with water will leak slightly and slowly relieve the rest of the pressure through the hole the pin made. There is no stored energy in the water because it is not compressed.

We have a heavy duty hydro chamber with blast proof glass on top so that we could film the test as well. We used a digital data recording system to record the water pressure at a high rate of speed so that we could see the exact pressure that the bottle would rupture or fail at. Unlike the water balloon analogy, we were taking these bottles to 3000-5000 PSIG before they would rupture so it was a little more violent than pricking a water balloon with a pin but still much less violent than doing the same failure with an expanding gas.

The point of the testing was to prove that the COPV designer's processes were controlled well enough so that a series of bottles manufactured with the same process would withstand the same pressure before failing. It was the first step in figuring out safety factors for the COPV's that he was building. The first tests were immediate successes. Not only were the failures all above predicted pressure, but they were very consistent as well. We tested some 20 bottles to failure at 3500 PSIG and they all failed within 100 PSIG of each other which was actually much better consistency than anyone was predicting.

What wasn't really predicted was the way in which the COPV's failed. Since we were trying to get exact data we were pumping the bottles up fairly slowly. The first bottle we ruptured exhibited some strange behavior we were not expecting. As we approached the predicted failure rate and slowed our pumping rate even more we heard a muffled popping noise followed by an immediate drop in internal pressure in the bottle. We were puzzled by this to say the least. The pressure stabilized but it had dropped some 75 PSIG immediately when we heard the popping noise. As we began pumping again we would see the pressure rise again but it would immediately fall back whenever the pumping piston retracted. Having done a lot of hydro testing over the years we suspected we were seeing some sort of tiny leak to cause the pressure to drop. Since water is not compressible even a small drop of water is leaking is enough to cause a significant and measurable drop in pressure. What didn't make sense is why the pressure stabilized again after it dropped.

We surmised that we may have found the pressure at which a thread or fitting was leaking but since the tank was under water there was no way to locate where it might be leaking. With the designers permission was decided to go ahead and see if we could overcome this leak with enough pumping pressure to cause the tank to fail. It took quite a few strokes from our pumping system and we had several more muffled pops followed by further drops in pressure but we eventually did rupture the bottle pretty violently. Once the carbon fibers gave way and the tank ruptured it looked very much like an exploded bundle of carbon wires. The aluminum tank underneath ripped violently open and frayed carbon resembling an angry porcupine angled sharply out from the breach in the tank.

It made interesting viewing on video and the data system captured the exact peak of pressure that caused the tank to rupture. As we looked at the pressure data more carefully later we could see the rises in pressure followed by the gradual drops after the popping noises started. We soon realized that the popping noises we were hearing were the individual carbon fiber strands popping in the bottom layers of the wrap. Each time a strand broke the aluminum tank swelled a little more as it was freed from the captured restriction of the composite overwrap material. We were effectively blowing the aluminum tank up like a balloon as the carbon fibers failed, adding space for more non-compressible water with each breakage. Eventually, enough carbon fibers failed so that they could no longer contain the swelling aluminum tank and the whole tank violently ruptured.

Later on when we filmed with high speed video we could actually see the tank lurch with each pop of a carbon fiber strand but by then we were well familiar with the failure mechanism of the tanks. The underlying carbon fibers break first and since there are so many layers of fibers it is quite impossible to see any change in the tank but the pressure trace sees the extra volume afforded by the resultant expansion in the form of decreased pressure.

After we completed a series of tests of water testing the designer suggested he needed to know how cryogen temperatures would affect the strength of the tanks. In other words, since we knew he could consistently predict their failure in water could he also consistenly produce the same results at cryogen temperatures? The end result of such design and testing would be to have a COPV that could be imbedded in a LOX or Liquid Hydrogen tank. The weight saving would be huge and the expense to produce such COPV's would be much less than a similar metal tank.

Rupturing a COPV at 3500 PSIG in Liquid Nitrogen (-320 degrees F) turned out to be a little more problematic than anyone planned. The first need was to get the tank chilled to LN2 temperature which took quite a bit of LN2 and was a slow process involving creating a tube that the COPV would fit inside. The tube would then be slowly filled with LN2 to chill the outer part of the tank to temperature. After this was accomplished we would fill the inner part of the tank with LN2, being careful to remove all compressible gas at the same time. This was accomplished by a high point bleed that we opened until we got LN2 out as we filled the tank from the bottom.

It is important to remove all compressible gas to minimize the stored energy that will be released when the bottle ruptures. While LN2 is like water not compressible we knew it would go through a rapid phase change once the tank ruptured. At 68 degrees F LN2 expands 694-1 as it changes for a liquid to a gas. In other words one gallon of LN2 instantly increases to the volume of 694 gallons when this phase change occurs. This phase change is almost instantaneous so we knew that when the tank ruptured we could see a very violent and quick phase change shock wave.

To minimize the already considerable explosive power we were going to produce we were very careful to keep the LN2 in the bottle in a liquid state. We knew that any LN2 that flashed to gas would then be compressible, thereby increasing the explosive power we were going to release when the tank ruptured. We performed this test in the abandoned back area of the test area at Marshall. We utilized some very large steel I beams to build a barrier around the test setup knowing we could direct the shock wave upward in this manner. We also ran all our control and instrumentation wires into a blast bunker on the bottom of one of the test stands so that we would be removed from the vicinity when the rupture occurred.

We put temperature sensors on the pump feed line into the COPV and planned to keep our pumping speed low enough so that the natural heat of compression of a pumping piston would not flash the LN2 to a gas as we pushed it at increased pressures into the COPV. We set up video to capture the explosion itself but the main data we were after was the pressure at which the COPV would rupture at LN2 temperature. The designer suggested it might actually hold more pressure at cryogen temperature as the carbon wrap fibers themselves would tend to shrink and more tightly hold the inner aluminum tank in compression.

As soon as everything was set and we had cleared the surrounding area of all personnel we began our process. It took quite a while to slowly chill the COPV so that we could cover it with LN2. The real problem came after we filled the COPV with LN2 and began slowly pumping the pressure up with a cryogen pumping cart. The heat of compression would quickly overcome the boiling point of LN2 and we would begin to flash to gas on the inlet line of the COPV. Our test design review board had set a hard temperature number barrier on this line that we could not go above as it would increase the explosive power of the tank failure considerably.

After several hours of pumping we were nowhere near the pressure we thought it would take to fail the tank because we were having to stop so frequently to allow things to chill back to liquid temperature. Unfortunately, every time we stopped the return to liquid temperature would also decrease the pressure in the COPV as the density dropped. It was a losing battle and we soon knew we couldn't gain enough pressure to fail the tank.

After more study we decided to better insulate the fill lines and move the cryogen pump much closer to the test article. We moved the massive steel I beams with a crane to get everything closer and set up for another run at failing the COPV. We improved the process such that we could get closer to the pressure we were looking to fail the tank but still eventually hit a point where we could no longer gain pressure and keep everything at liquid temperatures. The next step would have been to include vacuum jacketed lines and a lot of expense that no one had funds for so after a quick phone session with the COPV designer and our test design review board everyone concluded we would let the temperature creep up as much as needed to achieve rupture pressure. The designer needed data for a conference he had coming up and since we had cleared the test area of personnel we simply bought the risk of destroying some of our test equipment when the tank ruptured.

We knew both the liquid tube the COPV was chilling in and all of our safety barriers would force the blast wave upward when the tank ruptured so we were fairly certain that we wouldn't do a lot of damage to anything besides the tube and some of the attached tubing and instrumentation lines. Once we got underway again we got back to pressure fairly quickly and then began speeding up the pumping process as we watched the temperature and the pressure in the COPV climb. We knew we were creating a compressible gas bubble in the COPV to add to the phase change explosion that was coming but everyone had agreed it was an unavoidable risk if we were to meet schedule and budget.

The tank, true to the designer's suggestion, actually ruptured at some 400 PSIG higher than the same design had failed at in water. We got a beautiful pressure trace showing the same popping and swelling scenario we had seen in water. We also got exactly two frames of video on our normal speed video showing a veritable rocket rising on a plume of cold gas out of the liquid soak tube. It took a little while to find the remains of the COPV and some of our tubing still attached to it. It was some 200 yards away in a swampy area next to the barrier fence seperating the test area from a wildlife refuge.

It was quite an impressive audible blast. The same size and design bottles that we had been more or less harmlessly popping underwater had produced a titanic blast with the phase change and added compressible gas that filled the bottom 1/4 of the tank when it ruptured. We don't really know how high it went as it went out of camera view in two frames.

We later did similar destructive testing at Liquid Helium temperatures to simulate the pressure rating of a COPV in Liquid Hydrogen. The design and control process that the designer used in making these tanks was very good as all failures were both predictable and consistent across many samples. This designer later left NASA and branched out to form his own company that now produces and sells these COPV's to space flight companies. The only company that I know of that utilizes these COPV's in cryogen tanks on vehicles is Space Ex.

Going back to the data that seemed counterintuitive on the Falcon 9 that exploded mid-flight; the confusing thing for Space Ex and many others that looked at this data was that the Helium pressure in these COPV's dropped at the same time that accelerometers picked up a popping "sound" in the area of the COPVs. It then immediately returned back to "normal" range right before the LOX tank overpressurized and exploded. As I have detailed above this is exactly the same sequence we saw when we were testing these tanks to failure in our lab. The popping noise was the inner carbon fiber strands breaking. The drop in pressure was the aluminum inner tank expanding like a balloon.

On Space Ex's Falcon 9 there are several COPV's tied together with a common tubing manifold. If one tank were to start to swell as the composite fibers break the pressure would quickly equalize between it and the other tanks creating a return to "normal" pressure reading on the whole system. However, the increasing pressure would quickly break more composite fibers until the damaged tank failed explosively. The severity of the resulting expansion would be maximized by the fact that the LOX tank itself was very full at this stage as the second stage was not in operation. This would mean there was a very small ullage or gas bubble for compression in the tank leaving only incompressible LOX which would quickly rupture the LOX tank.

Both Space Ex and the supplier of the COPV's will quickly explain that the COPV's have been thoroughly tested to much higher pressures than those they saw on flight. I will also attest to the fact that the designer of these COPV's has extensive data showing how good his design and manufacturing control processes on these bottles are. I have no doubt that these bottles will not fail under design pressure when handled properly.

Going back to my original discussion of COPV's I also explained that COPV's suffer from a couple of weaknesses as flight pressure tanks. The first is cycle limits, although this is probably not a problem in this usage. The second is the fragile nature of the exposed carbon fiber shells themselves. Each small carbon fiber is glued and interlocked in an intricate helical pattern to wrap and contain the thin aluminum tank underneath. Since the weight of the carbon fibers is minimal it is both cost effective and weight efficient to build in large overpressure ratings for COPV's. A tank that much contain 5500 PSIG can easily be manufactured to safely contain much higher pressures before failing. I have no doubt that the tanks flying on Falcon 9 are indeed rated and tested to much higher pressures at LOX temperatures than those they actually see in flight.

Musk has been especially adamant that the Carbon shells will not fail at pressure. No doubt he has seen a lot of data from the manufacturer proving this to be true. However, the fragile nature of the carbon fibers themselves require very careful handling from manufacture to test to installation to assure that they are not damaged before usage. Unfortunately, there is evidence that this has not been the case in the usage of these tanks.

Previous to flight Space Ex is in the habit of taking photographic evidence of all phases of their Falcon 9 assembly process. These pictures are known as "closeout" photos and stand as visual proof that all nuts are lockwired and all tiedown and cabling is carefully attached in the vehicle. I have not personally seen these pictures but some of the review teams I work with have. In at least one of these pictures the photographer himself is standing on the support struts that the COPV's are mounted to. No flight hardware is rated to be used as a standing platform. It is an egregiously bad practice for anyone to do so under any circumstance; either during fabrication and assembly or at any other time. The fact that someone is indeed seen standing on something as delicate as the COPVs which are inherently prone to serious and debilitating damage from relatively minor exterior mechanical force is inexcusable. The fact that people within Space Ex have seen and distributed these pictures seems to point to the fact that there is little or no understanding of the fragile nature of the COPVs.

While this is probably a little long winded it does plausibly match the actual data from the Falcon 9 that exploded in mid-flight. Could the same thing have happened to the Falcon 9 on the pad at Kennedy. Preliminary evidence suggests that it was a COPV failure in the exact same location that led to this explosion as well. There are also some necessary precautions that must be taken when initially pressurizing COPVs with gas that are not necessary with metallic tanks. I will go into that in my next post.