Appendix A

Operating Scenario

GPS/CDU Project for Wild Blue Yonder Technologies

Wild Blue Yonder Technologies Inc (WYBT) is a general holding company whose line of business is tailored to high-tech holdings. Wild Blue Yonder Technologies various subsidiary companies are maintained as one coordinated business from offices in New York City. The centralization of policy and planning direction at one location has historically produced higher revenues, profit margins, and customer satisfaction. The necessary degree of coordination is enabled by a global, enterprise network that is managed from the New York location.  That network provides secure telecommunications capability with embedded firewall protection, multi-carrier cellular access options and automatic access point database updates for all connection types. It enables access to the enterprise’s applications from any location on an as-needed basis. The network also provides integrated, any distance, seamless connectivity to WBYT’s centralized information resources.

WBYT’s holdings are concentrated in advanced technology products and services. Two closely held subsidiaries deal exclusively with the Federal government. The line of business of one, which is based in Gaithersburg, Maryland, is R&D and manufacture for advanced capability components for the F 16 Fighting Falcon and F 18 Super Hornet. The other, based in Jacksonville deals in R&D in target acquisition and fire control systems for Army helicopters. There is also a manufacturing facility in Detroit. That facility builds Leopard tanks for the Canadian Army under license from the German government. Other close holdings in WBYT’s empire include a commercial electronics R&D facility in Corvallis. The Corvallis facility also does contract work for the Idaho National Laboratory. In addition to the closely held corporations, there are loosely held electronics manufacturing, or service holdings in Pittsburgh, Houston, Des Moines, Sioux Falls, Denver and Bozeman. These facilities serve the consumer high-tech industry.

Finally, there are a number of loosely held international corporations in India, Australia and across the Pacific Rim, all concentrated in advanced technology. All computer services for that region are provided over a public/private VPN, which is maintained for that area in Singapore. The Singapore data center is actually owned and operated by WBYT, as part of the company’s global VPN. The VPN itself is maintained out of the New York office.

According to WBYT’s charter, the primary business goal of the Company is to utilize the global marketplace to provide high quality technology components at the lowest price possible price.  Wild Blue Yonder Technologies entered the market knowing that the ability to closely monitor its operation and deliver competitive business information quickly was going to be a prerequisite to its success, particularly in the integration and reuse of COTS products. In essence, its entire business model was based on the presumed ability to do that. In fact, since information was the key to company survival, that mission was laid out even before the technical capability for achieving it was in place.

Wild Blue Yonder Technologies information processing operation delivers information and services to its various subsidiaries in two ways: hosted and embedded. The hosted model removes the burden of maintaining on-site data acquisition and management functions from the facility’s operations managers, while ensuring a secure and scalable worldwide environment. The embedded model allows each local facility to operate and maintain its own IT infrastructure, which is tailored around WBYT’s enterprise systems to support that subsidiary’s specific line of business and business operation.

Company Organization Overview

Because it is focused on the development of very advanced software products WBYT gets most of its business from the Pentagon. It utilizes a well-defined, flexible process for planning for and developing this software. It is also known as an innovative place to work. WBYT is operating under a pressing NIST 800-53 mandate and the only process holding up the assessment is the Risk Management area. As such, the WBYT management would like you to implement a robust and persistent risk management process for its supply chain.

Early software project planning is stressed at WBYT, and project plans are developed to integrate effectively with the other engineering plans within each project. There is strong informal communication among all the engineering disciplines, and a single program manager manages each new project from an integrated system view. Software estimates are derived through expert analysis and documented for use throughout the project’s life. These estimates are backed up with outputs from estimation tools that are used to provide a “reality check” to the experts’ initial idea. Actual project data is retained to support a cost estimation improvement effort under way at WBYT but it is not used in a formal feedback sense.

Software project management metrics are used to provide visibility into project performance at the project level. When performance deviates from the initial plans, the project manager is responsible for either making changes to the way the project is being handled (in order to bring the project back into conformance with the plan), or re-planning. Software subcontracts are managed using a set of defined policies and procedures. Software requirements, design, and code inspections are used to support development. Defect metrics from the inspections are maintained. Other product related metrics are identified and maintained for each development effort to help keep reasonable visibility into the development effort. These metrics also are used to support software project management and risk assessment. The only problem is that all of this takes place at the project rather than the organizational level. The Program Manager and upper management never see the results of this extensive measurement process.

The review culture at WBYT is not well developed. However, assurance is primarily defined as testing the code. There is no software configuration management at any level in the supply chain. A SEPG team of engineers and managers from the software engineering organization are responsible for keeping the approved software engineering processes up to date, and identifying new opportunities for improvement. This team reports to the manager of software engineering and to the corporate vice president of engineering. The vice president of engineering maintains a keen interest in the software engineering processes for the corporation. The manager of software engineering and the vice president of engineering are responsible for providing quarterly reports to the company president on the state of software engineering and software assurance process improvement. The problem is that most of this reporting up and down the chain of command is in the form of rumor rather than objective data.

COTS and GOTS

WBYT uses COTS and GOTS software when possible, but it does not hesitate to build its own components when necessary or to mitigate risk. Humongous Holdings chief architect and the CIO are both former employees of a major Web search engine site/content provider. In four years back in the late 1990s, they watched that provider’s usage go from 45,000 to 45,000,000 page views per day. With millions of people using the system, they learned very quickly to take whatever security precautions needed to avoid being awakened in the middle of the night with a business-threatening problem.

WBYT’s management’s major concern about COTS products centers on the ability to ensure its security.  That was particularly true when targeted bench-checks found Trojan code embedded in products that were acquired from an overseas source. Thus, when security is essential and the source of the code is in doubt, WBYT will not hesitate to build the necessary components in-house. WBYT’s rule of thumb is that if the function is unimportant, COTS will do. If there’s an actual or de facto security requirement for some aspect of the system, the COTS product will have to be proven secure. Otherwise, that component is a strong candidate for in-house implementation.

Operations

The “Operations” practice area is concerned with the day-to-day running of the assurance operations, the primary mission is to ensure that all new systems are, built, launched and deployed correctly. That implies extensive use of all of the software and quality assurance and testing practices as well as security assurance. In order to identify security problems the core security team, which is based in New York, holds a quarterly “security summit” event every month involving the security managers from all of the subsidiary sites and any major sub-contractors. The aim of these summits is to address any operations-level concerns as well as ensure that all projects are satisfying their security metrics. The objectives of these meetings include

• Coordinate the dialog between owners of front-end components and owners of backend components. The outcome could be a written description of interface roles and responsibilities, which is captured in a formal organizational agreement.

• Resolve any communication or operational conflicts.

• Make sure that everyone is on track. In the words of one participant, they “will stop heaven and earth to resolve issues.” The idea in these meetings is to do what it takes to make sure that everybody is on the same page.

  Data Collection, Metrics, and Tracking

  WBYT uses security metrics as a key-planning tool. For each protected item in its inventory, WBYT measures the:

• Projected number of threats with impact (as likelihood percent)

• New threats identified (as frequency)

• Actual number of incidents (as count)

• Time/Date of incident

• Cost of incident

• Violations involving staff (as incident count)

• Number of software defects reported (as count)

• Number of software defects repaired (as percent)

• Cost of repair

   

  These measures help WBYT determine how to allocate its security resources. The measures also ensure that Humongous’s security operation is properly targeted and monitored. The measures also help to alleviate executive level prioritization battles by establishing a historical baseline for each item. That baseline allows WBYT to make decisions about expansion of its holdings and other strategic directions.

The GPS/CDU Project

For this project, WBYT has contracted with the Air Force to upgrade the F-16F aircraft. Specifically, the Air Force wishes to update the navigation system with Advanced Global Positioning System (GPS) capability that will allow that fighter-bomber to pinpoint targets down to the foot, even in overcast and nighttime conditions. The GPS model chosen for this aircraft has been used in a similar application for the fire control system for the AH64D Apache Longbow helicopter, so it is considered to be off-the-shelf (COTS). However, because it’s an Army item, modifications to the software will be required in order to integrate this GPS system into Air Force aircraft. Revisions will need to be written (and/ or modified) to support the following needs:

In the case of the GPS/CDU Upgrade itself, the products are the onboard GPS Interface and the pilot CDU. This project can also be decomposed into the constituent management processes. In this case the major processes are project management, COTS acquisition and integration, COTS interface and software assurance support, software customization and risk management, supplier and software qualification, and post-development support.

The requirements for the GPS interface are well known. The current navigation system and interface software is written in Java, and due to the limited nature of the changes to the navigation software, no change in language will be required. ISO 12207-2008 will be used as the software development and documentation standard for this acquisition. NIST 800-30 and NISTIR 7622 will be used to manage the risk portions.  ISO 15408 might also be consulted to describe any product certification needs. The final product will also have to satisfy all relevant elements of NIST 800-53

The contractor that developed the GPS system for the Army will make the GPS software modifications on a subcontract to WBYT.  Consequently their supply chain will have to be vetted along with that of WBYT. WBYT will provide a specification of the modification requirements to all subcontractors however any subsequent product and process assurance up and down the supply chain will be up to the individual subcontractors by contract. This will reduce the cost associated with modifying this aspect of the system.

WBYT has determined that besides any COTS acquisitions about 9,000 SLOC will have to be developed and/or modified to implement this new capability. The condition of the existing navigation software is not known and the customer wants formal documentation, so it was assumed in the planning for the bid that the relative level of productivity would be at about the same level as that for a new development. The Air Force (acquirer) has contracted to supply a System/Subsystem Specification (SSS), a System Design Description (SSDD), and an Operational Concept Document (OCD). The acquirer will modify the OCD during development to reflect the changing needs of the system. WBYT will be responsible for the following technical documentation:

The developer will also be responsible for the following management and support documentation:

Resource Requirements

Based on the preliminary project plan, the software effort will take approximately 3.25 person-years over a period of 12 months. The development, test, and technical documentation effort will be approximately 2.25 person-years; the remaining 1.0 person-year will be dedicated to software project management, the support documents, SQA, SCRM and SCM. User documentation will be produced as part of the new CDU development effort. The GPS subcontractor will develop the SRS and SDD for the GPS from existing materials created for the Army.

The pilots for this aircraft dislike the CDU and want a replacement. The current CDU has a very small keypad, and the display is only 4 lines by 40 characters. The pilots would like to at least double the display capacity, increase the display visibility, change the keypad, and add new query capabilities. The pilots want an opportunity to help determine the requirements for the CDU and to have significant inputs to the changes to the user interface. Since the pilots want so much input to the new CDU, WBYT feels that prototyping will be useful to support the user interface development. Also, it is not known how many new capabilities will be added to implement the pilots’ query requirements. WBYT is assuming that the new CDU will have all new software. The Air Force wants the new CDU to be implemented in Java, and the existing code is in C++ so it is assumed that there will be no software code reuse in that module. ISO 12207-2008 also will be used for the CDU development, and as with the GPS integration, the Air Force is requesting significant amounts of formal documentation.

Based on WBYT’s knowledge of the existing CDU and a rational appraisal of the possible new requirements, WBYT assumes that the new CDU will require about 20,000 SLOC to implement. Given the unknown nature of the new requirements, this is a soft estimate. The customer wants the new CDU to be a significant improvement from the current version, so there may be some flexibility to re-negotiate the terms of the contract when the first sets of prototypes are completed and approved by the pilots. In both parts of this upgrade project, software supportability is a primary concern. Thus, planning for the software transition to the Government and for Post Deployment Software Support (PDSS) is stressed in the acquisition. This may add labor to the existing software estimates from WBYT.

Project Characteristics

Here is some additional information that you might need:

The areas for concern are human safety, the level and formality of documentation, compliance with all relevant regulations and standards and the degree of change in philosophy from previous standards, lack of formal reviews, and the unknown complexity that could exist in the interface code. On examining the project characteristics, the project manager notes the following regarding these issues:

Here are some additional things to consider

1. The  integration needs to be carefully controlled throughout the project

2. Review requirements must allow for in-process reviews. These should be fully documented.

3. The metrics practices and the means for external distribution of metrics data has to be specified. This should be fully documented.

4. Where no reviews exist, those reviews will need to be supplied to cover a specific process area.

5. All identified risks must be mitigated