Common Test & Evaluation Analyses
Defense systems are complex systems that contain a variety of software and hardware components. They consistently push the limitations of scientific understanding, synergizing new and unique interfaces among software, hardware, and users. Testing these complex, multi-functional systems, especially when you consider the range of system types that must be tested, demands both a structured and flexible analytical process. Thorough testing of defense systems must ensure that they will work in a variety of complex combat environments and scenarios.Characteristics of Test and Evaluation Goals
To ensure thorough testing, defense system testing occurs in multiple phases – developmental and operational. Both use similar analytical tools, but emphasize different goals.Developmental Testing
Developmental testing focuses on understanding system capabilities and limitations while still in the development phase. This often means tests are conducted on critical subsystems before they are integrated into the full system. Additionally, in contrast to operational testing, developmental testing often takes place in a more controlled environment, making it the best time to use DOE to learn about key reasons why system performance changes as a function of test conditions.Key focus areas for developmental testing include:
- - System Performance
- - Reliability
- - Interoperability
- - Cybersecurity
Operational Testing
In contrast, to developmental testing, operational testing focuses on the effectiveness and suitability of systems when used in the operational environment, by operational users. Key components of effectiveness often include mission accomplishment and system performance. Suitability, on the other hand, often relies on the assessment of reliability, maintainability, and availability as well as Human Factors or Human Systems Integration measures (e.g., workload, usability, training adequacy).Key focus areas for operational testing include:
- System Effectiveness
- - Performance in Operational Environment
- - Support of Mission Accomplishment
- System Suitability
- - Reliability, Availability, & Maintainability
- - Usability, Workload, other Human Factors
