Boeing Analyses How It Was Judged by US Air Force
(Source: Boeing Co. via Seattle Post-Intelligencer; posted March 10, 2008)
ST. LOUIS --- Following the Air Force's KC-X decision announcement, press articles have appeared quoting aerospace experts who purport to have insights into why the KC-767 was not chosen. These articles allege that "Northrop Grumman's victory was not a close outcome" and that "Boeing didn't manage to beat Northrop in a single measure of merit."

Nothing could be further from the truth.

Factor 1 -- Mission Capability
-- Boeing scored "Blue (Exceptional) and Low Risk" in this area – the highest possible rating in the most critical "factor" in this competition.
-- The Air Force assessed Boeing as meeting or exceeding all Key Performance Parameters (thresholds and objectives). Indeed, the Air Force evaluated Boeing as having significantly more strengths (discriminators) than the competitor.

Therefore, it follows that Boeing 1) received the highest rating possible, 2) met or exceeded all KPP thresholds and objectives, and 3) was graded as having significantly more strengths than the competition

Factor 2 -- Proposal Risk
-- Boeing's proposal risk was rated "Low"
-- Surprisingly, the competitor was also rated as low despite the high risk associated with its evolving multi-country, multi-facility, multi-build approach as contrasted with Boeing's integrated approach to design, build, and certification in existing facilities with experienced personnel

Therefore, it follows that Boeing 1) was low risk, 2) had an integrated and lean build approach, and 3) the competition should have been assessed greater risk for its complex and unproven multi-country build approach.

Factor 3 -- Past Performance
-- Boeing's past performance was rated "Satisfactory"
-- Northrop Grumman/Airbus was also rated satisfactory, despite having no relevant tanker experience and having never delivered a tanker with a refueling boom
-- Press reports indicate that some of the most relevant programs for Airbus (the KC-30 for Australia and the A-400M) are both significantly over cost and behind schedule.
Therefore, it follows that Boeing 1) had satisfactory past performance, and 2) relevant Airbus programs like the Australian KC-30 tanker and the A-400M are struggling.

Factor 4 -- Cost/Price
-- As determined by the RFP, "Most Probable Life Cycle Cost" (MPLCC) was the only measure of cost to be assessed
-- The Air Force described the cost visibility information Boeing provided as "unprecedented" and rated Boeing's MPLCC cost "Reasonable," "Balanced," and meeting "Realism" criteria – all the highest ratings a competitor can receive
-- As recognized by the Air Force itself in 2002, the significantly bigger A-330 would demand a greater infrastructure investment with dramatically lower operational effectiveness

Therefore, it follows that 1) Boeing's MPLCC was judged by the Air Force to be realistic, 2) Boeing's submitted MPLCC were significantly lower than the Air Force adjusted MPLCC costs and, 3) the Air Force adjustments to Boeing MPLCC costs effectively deprived Boeing of the benefits associated with its integrated in-line production approach.

Factor 5 -- Integrated Assessment
-- The model used by the Air Force to judge tanker "fleet effectiveness" was developed and is maintained by Northrop Grumman
-- The mission scenarios and operational constraints to be used with the model issued in the draft RFP to judge tanker "fleet effectiveness," were based upon the 2005 Air Mobility Command "Mobility Capabilities Study" (MCS).
-- Before and after the RFP release, changes to the model's parameters occurred so as to allow a "greater variety of aircraft to be considered" – in essence to allow larger aircraft to compete. However the Air Force promised that it would tie the numerical output of the model back to real-world constraints by weighing "insights and observations."
-- The inherent complexities of the model have made its results inconsistent and un-repeatable and its overall operational relevance questionable;

Therefore, 1) Northrop Grumman's experience with the model was an inherent advantage, 2) changes were made to ensure Airbus' larger aircraft worked in the model, but there is little evidence that the Air Force used "insights and observations" to tie the model back to real world operational constraints and 3) the model's accuracy and relevance are debatable.

Boeing submitted a strong and extremely competitive proposal. In assessing the critical factor of Mission Capability, Boeing was given the highest ratings and evaluated by the Air Force as having significantly more strengths (discriminators) than Northrop Grumman/Airbus.

The Air Force modified the Northrop Grumman analytical model before and after issuance of the RFP to enable competition and to allow a larger tanker to compete. In the end, the "leveling" of the competition and subjective assessments of the two proposals seems to have led the Air Force to select a larger, more expensive and operationally limited KC-30 tanker despite the fact that both Air Force requirements and the KC-X RFP call for a medium-sized tanker to replace the KC-135.

Boeing Fights Back: How It Plans to Prevail
(Source: The Lexington Institute; issued March 12, 2008)
(© Lexington Institute; reproduced by permission)
By Loren B. Thompson, Ph.D.

When Boeing executives heard last week that they had failed to beat Northrop Grumman in any of the five selection criteria for the Air Force's future aerial-refueling tanker, they were incredulous.

Their reaction turned to anger when they were debriefed on the decision by Air Force officials. Although the debrief confirmed that they were beaten on four of five measures (and tied on the fifth), the company detected numerous errors in the process.

Boeing has now embarked on a two-part strategy to overturn the decision. First, it will file a formal protest with the Government Accountability Office alleging procedural and analytical errors in the awarding of the contract. However, since any GAO determination in its favor would be advisory rather than binding -- the Air Force can ignore the finding -- Boeing will not oppose legislative efforts by its backers in Congress to nullify the Northrop win.

Here is Boeing's internal assessment of Air Force mistakes made on each of the five selection criteria...

1. Mission capability measured the performance features, support capabilities and technological maturity of the competing proposals. Boeing says it satisfied all stated requirements, tying or surpassing Northrop Grumman in most (for example, it was rated higher on survivability). Northrop won overall due to greater fuel and cargo carrying capacity, but Boeing says that deviated from Air Force assertions that the service was seeking a medium-sized tanker.

2. Proposal risk assessed the degree of danger that the two teams would fail to execute as promised. They tied on that measure after Boeing lengthened its original development schedule. However, Boeing argues the Air Force failed to accurately assess the risk of Northrop's plan, which involves building major assemblies in several countries and then integrating them in a new facility in Alabama.

3. Past performance compared the success of the two teams on programs similar to the future tanker. The Northrop Grumman team was rated higher, but Boeing says their competitor has faltered on programs such as a new Australian tanker and the A-400M cargo plane. This measure was originally scored by Air Force evaluators as a tie and then adjusted to favor Northrop for reasons Boeing finds questionable.

4. Cost/Price is the area where Boeing always expected the Northrop team to fare best, because European partner EADS is not subject to the same profit pressures as Boeing. But the key pricing metric was "most probable life-cycle cost," and Boeing executives are certain their smaller plane costs less to fuel and operate. They also contend the multi-site, multi-country assembly plan for the Northrop plane is intrinsically more expensive.

5. Integrated Assessment rated the competing planes in a warfighting scenario using a complex analytic model. Boeing believes Northrop won this measure because the model was originally developed by Northrop, and was not an accurate reflection of real-world conditions. It also contends changes were made in the model to permit Northrop's participation in the competition, but that little effort was made to look at actual operational experience in assessing the planes.


prev next

Official reports See all