Prepared Remarks by Philip E. Coyle, III
Before the Standing Committee on National Defence
House of Commons
Ottawa, Canada, March 29, 2007
Mr. Chairman and Members of the Standing Committee on National Defence, I very much appreciate your invitation to appear before you today to support your study of military procurement and associated processes, including the tendering process and the establishment of capability requirements.
Ladies and Gentlemen of the Committee, I currently am employed as a Senior Advisor to the non-profit Center for Defense Information, a division of the World Security Institute, a Washington, D.C.-based national security study center. To help insure our independence, the World Security Institute and the Center for Defense information do not accept any funding from the Federal government, nor from any defense contractors.
From 1994 to 2001 I served in the Pentagon as Assistant Secretary of Defense and Director, Operational Test and Evaluation. In this capacity, I was principal advisor to the Secretary of Defense and the Undersecretary of Defense for Acquisition, Technology and Logistics on test and evaluation in the DOD. I had OSD OT&E responsibility for over 200 major defense acquisition systems.
From 1959 to 1979, and again from 1981 to 1993, I worked at the Lawrence Livermore National Laboratory. Over those 33 years I worked on a variety of high technology programs, and retired from the Laboratory in 1993 as Laboratory Associate Director and deputy to the Director.
In my current capacity at the Center for Defense Information I am called upon to provide independent expertise on various defense matters. I have over 30 years of experience involving U.S. defense systems and equipment.
My remarks today will be based on those experiences and may not apply to the situation in Canada which may be quite different from that in the U.S.
The Current Situation
I want to note from the outset that many U.S. military acquisition programs work very well. You never read about them in the newspapers.
And they provide the user, the warfighter, with the intended capabilities.
However, there have been some disturbing trends:
- In some recent years, 80% of Army systems did not achieve 50% of their required reliability in operational testing.
- Not long ago, 66% of Air Force systems had to halt operational testing because they were not ready.
- The Navy has had to deal with such difficulties also. In 1992, only 58% of Navy systems undergoing operational testing to support a Milestone III decision were successful. The Navy instituted several changes to their acquisition process - including not going into operational testing before they were ready - and, a few years later, their success rate was up to 92%.
More generally, today there is concern about findings ways to reduce technology risk in U.S. defense acquisition programs which too often overrun their costs and schedules.
Four Critical Problems
When such problems arise it is usually because of a general lack of realism, a lack of realism which manifests itself in four basic ways. Incidentally, none of these observations are original with me. Many others have noted these same issues for years, and these topics are the subject of regular conferences and symposia.
1. Unrealistic Requirements
Naturally, we want our soldiers, sailors, airmen, and Marines to have the very best equipment. And they want that too. The users, the warfighters, also want more capable systems, which can translate into multi-mission systems.
This leads to more complicated, multi-functional systems, often with computers and sensors working together for information fusion. These days practically everything has a computer in it; for example, the cancelled Crusader howitzer program had roughly a million lines of code in its computer. Some people are surprised that a howitzer would need that sort of computational power, as it is approaches that in a modern jet fighter aircraft.
In the U.S., the technical challenges that must be overcome to achieve effective, multi-function systems are regularly underestimated.
2. Unrealistic Costs and Schedules
It is not uncommon for the U.S. Department of Defense to have unrealistic expectations for cost and schedule, as well as performance. Sometimes this originates in proposals first put forward by industry. To make new proposals attractive, U.S. defense industry may overstate what can be delivered and how inexpensively. Whether driven by contractors or by the government itself, this leads the contractors to "buy-in" in order to be competitive.
This often is caused by a failure by both the government and the contractor to fully understand and address the technical challenges in a program early. When the technical challenges have not been candidly identified, the costs and schedules to solve these problems can overrun by billions of dollars and many years of delay.
Also, with the intent of saving time and money, sometimes the military departments or defense contractors turn to "commercial off-the-shelf" or non-developmental items (COTS/NDI). Usually these items are really not “on the shelf,” commercial or otherwise, and if they are often the designer never contemplated that the commercial product would be put to military use in a harsh military environment.
3. Unrealistic Contractual Environments
Too often, the U.S. Department of Defense goes into highly complex, technical programs expecting the contractors to deliver under firm, fixed price contracts. Even if they are not firm, fixed price, many contracts are structured with no incentive to continue development to improve the system, and every incentive to get into production as soon as possible. Also, the contractual environment for these contracts often requires defense companies to make unrealistic bids simply to be competitive.
You've heard the phrase, "you can make it up in production". Later if production quantities are cut, which they often are, that further reduces the prospects for profit.
4. Preparing for Realistic Operational Environments
Sometimes defense acquisition programs underestimate the operational environment. This can be operational environments such as bad weather, but it also can be the stresses of battle or operational loading. For example, computer systems may be loaded much more heavily in battle than in the laboratory.
Sometimes acquisition programs do not prepare adequately for operational tests, which - by definition - will be operationally realistic. Complex systems that have done well in the laboratory, sometimes do not perform well in realistic operational tests.
Here are ten things the Parliament may already be considering that would help solve such problems.
1. Parliamentary Oversight and Review
First, increase Parliamentary oversight and review. There are many ways to do this, but generally when the U.S. Congress maintains closer oversight and review U.S. soldiers get better, more effective equipment, sooner and cheaper. In Canada - as well as in the U.S. - this requires legislative staff with the kinds of technical backgrounds needed to fully understand what the contractors are claiming, and how likely it is that those claims can be realized.
Another important oversight activity is to track the ongoing costs for military procurements, especially when they are overrunning their costs and schedules significantly, or producing less than desired results. In this regard, I would caution against the U.S. practice of regularly re-baselining costs and schedules. This makes it very difficult for the U.S. Congress to keep track of how much the costs have grown or the schedule slipped. If the contractor building your new house kept re-baselining his costs and schedules, and didn’t report on their cumulative impact, you could end up with a very expensive house.
The Parliament could also turn to the Auditor General for help in this regard, as the U.S. Congress often turns to the Government Accountability Office (GAO). However, once again, the Auditor General must have people with exceptional technical backgrounds to do this well, as must the GAO.
2. Competition in contracting
Second, sustain competition as long as practicable in the procurement process. In military procurements, as in every day life, competition nearly always produces a better product, sooner and at a lower price. Some will argue that competition is expensive as it requires sustaining two contractors when one would do. But sole source contracts in America often overrun their costs and schedules far more than it would have cost to have had competition. Also, sole source contracts often under deliver on promised performance. One of the benefits of competition is that it provides the government with a mechanism, sometimes otherwise lacking, for understanding what is realistic and what is not, what is likely to be delivered, and what is not.
One option is to structure procurement outcomes that encourage sustained competition, and that are not "winner take all". For example, contracts that result in a 60% leader and 40% follower, or 40% to two successful bidders with 20% more for the more competitive contractor later.
3. Requirements and Requirements Change Control
Third, provide a mechanism such that significant new requirements, as well as significant requirements changes, go back to senior defense officials and perhaps to the Parliament for review and approval. Too often user requirements are changed without senior level review of, or involvement in, the consequences of new requirements. This is sometimes referred to as “requirements creep,” and can be a significant factor in causing new military systems to be come bogged down in complexity. What will these new requirements do to the overall cost or schedule? Will these new requirement actually interfere with other higher priority needs? Is the technology in hand to meet these new requirements, and if not, when is it likely to be in hand?
And, if you do not already do so, make development costs reimbursable so that when requirements are changed there is a mechanism to pay for the development needed to achieve desirable new requirements.
4. Commercial-Off-the-Shelf/Non-Developmental Items (COTS/NDI)
Fourth, don't consider a product to be COTS/NDI unless you can truly buy it off the shelf, it comes with a users manual, and it has been tested in the same environments in which it is to be used by the military. It’s very popular these days to buy something for our militaries that is commercially available off the shelf, and that is a fine thing to do so long as the item really is off the shelf, and actually was designed with realistic military requirements in mind. But too often, it is not commercially available, and was not designed for the stressful environments in which military equipment must operate. Examples can range from small items like computer chips to large systems like the C-130J which was marketed as a COTS/NDI item.
5. Pay attention to contractor incentives
Fifth, pay attention to contractor incentives. To take an example from the United States, if defense companies are not allowed to make a reasonable profit for doing development before production, they will have less incentive to do it. Until the fall of the Berlin Wall, in the U.S., development was allowed to be up to 90% cost reimbursable. Today profit from development is discouraged, leading defense companies to want to get into production as soon as possible whether they are ready or not.
6. Avoid unrealistic expectations in defense contracts
Sixth, avoid placing unrealistic expectations in defense contracts. For example, avoid firm, fixed price contracts for high technology, high risk programs. If you are requiring state-of-the-art high technology systems, it may not be realistic to expect defense contractors to deliver those results under firm, fixed price terms. Considerable development may be required. Also avoid putting defense contractors in cutthroat, "unconditional surrender" competitions where they have to "buy-in" or "low-ball" to stay in the defense business.
7. Assess Technology Readiness Levels
Seventh, use assessments of Technology Readiness Levels as a tool to better understand the technological difficulty of new proposed new military systems. Here I recommend the reports put out by the U.S. Government Accountability Office on the use of Technology Readiness Levels (TRLs) as a tool to better understand the readiness of new technology for production. The failure to identify and address technology hurdles is a major cause for subsequent cost and schedule overruns in U.S. defense acquisition programs. The GAO has put out a series of reports on “Best Practices” for defense acquisition and procurement, and the DOD is beginning to adopt these recommendations.
8. Recognize the importance of government expertise
There is really no substitute for a well-informed customer, and when the customer is the government this requires government expertise. The trend in the U.S., and perhaps here in Canada as well, is to give over to private industry nearly complete control for the management of major U.S. defense programs. This can be through the use of very large prime contracts, or through “Lead Systems Integrator” (LSI) contracts. The phrase, “Total System Performance Responsibility (TSPR) has been used to describe this acquisition approach. With TSPR and LSI taken to an extreme, the government itself no longer has the wherewithal to understand or manage complex military procurements, becomes solely dependent on the lead contractor, and is prevented from properly overseeing the expenditure of taxpayer dollars.
9. Examine the status of major new systems early and often.
Ninth, examine the status of major new systems developments early and often. My successor in the Pentagon has pointed out that the cost of programs is generally driven by decisions that are made in the initial 10 to 15 percent of a program. If the government puts more emphasis on the early phases of the development process, that can avoid much more difficult problems later. These early assessments would include realistic examination of the state of technology required to achieve an effective system. Of course, to do this the government needs its own expertise, as mentioned above. The U.S. Department of Defense did this well in the case of the C-17 aircraft, resolving early problems, and leading to an effective strategic lift procurement program.
This recommendation also includes getting independent test and evaluation advice early. Among other things, this means funding your military personnel who are involved in testing new military equipment so that they have the people, equipment, and money to participate in acquisition programs from the very outset. Waiting until an item is in production is usually too late.
10. Anticipate support and sustainment costs
Tenth, deal with what a former U.S. Under Secretary of Defense for Acquisition, Dr. Jacques Gansler, calls the "Death Spiral". Often the sustainment costs for U.S. military systems far exceed their first costs. We often can see that these systems will be expensive to maintain early in development. Sustainment ought to become a regular part of the evaluation of proposed new military systems so as to identify sustainment issues earlier and correct them before they become a burden to operational forces in the field.
Mr. Chairman, this concludes my prepared remarks. As mentioned at the outset not all of these suggestions will be appropriate for the Canadian government. For one thing, the scale of most Canadian military procurements is different from that in the U.S. However, the trend in defense procurement worldwide is towards, larger, more complex and more costly systems that involve advanced technology, computers, and software, and these big systems are difficult to manage.
Thank you very much for your attention. I would be pleased to take any questions you might have.