This section presents several case studies of defense system tests requiring common T&E analyses. Evaluators testing similar systems or systems with similar response variables may follow the links to learn more about these statistical applications and to see how statistics are used to characterize systems.
Detection Time: A-RCI Sonar Processor
A-RCI is a submarine sonar processing system. Testing this system involves recording the response variable of detection time. More specifically, it is the time it takes an operator to detect a submarine once it has become visible on the A-RCI system screens. The analysis of detection time involves lognormally distributed continuous data as well as right censoring, two techniques that may be of use to evaluators who are testing similar systems or response variables.
Read more>>
Detect vs. Non-Detect: Q53 Counterfire Radar
Q53 is a radar system designed to detect artillery, mortar, and rocket fire in order to estimate the origin and destination locations. In addition to detection time, the probability that a target will be detected is an important aspect of detection system performance. To assess probability of detection, evaluators recorded whether or not Q53 detected each projectile that was fired. Analysts used logistic regression to convert this binary detect/no-detect data into probability of detection and estimate the impact of several factors on this probability. Logistic regression is a powerful analysis to characterize any system that produces binary data representing two discrete outcomes (e.g., Hit/Miss, Success/Fail).
Read more>>
Replacing Binary with Continuous Response Variables: JCAD
Continuous data provide the same information as binary data while augmenting it with rich details. Moreover, they do so with greater efficiency, as fewer test resources are required to provide the same level of information as their binary counterparts. The following example pertains to testing the Joint Chemical Agent Detector (JCAD; see Figure 1). It compares the required sample size as well as the results for the two different detection measurements: continuous (time to detection) and binary (detect/non-detect).
Read more>>
Reliability, Availability, & Maintainability: CAC2S Command and Control
The Common Aviation Command and Control System (CAC2S) is used to command, control, and coordinate air and ground operations. Reliability, availability, and maintainability of the system are essential to ensure that operators have situational awareness and assets can be controlled. Reliability of CAC2s was derived from data on operation time as well as number and type of failures. Two metrics of mean time between failures and confidence intervals were calculated and compared against the reliability requirement. Maintainability was estimated
These analyses indicated limitations, potential inconveniences, and areas of improvement.
Read more>> (Coming soon)
Workload: DCGS-AF
This section will highlight special considerations for analyzing survey data. It will focus on utilizing demographic information as covariates or factors and accounting for repeated measurement of human subjects in a repeated measures ANOVA or Generalized Linear Model when data is missing.
Read more>> (Coming soon)
The quality of human-system interactions is a key determinant of mission success for military systems. Often, testers evaluate human-system interactions using solely survey instruments, excluding other methods entirely. Multi-method approaches are more comprehensive than single-method approaches and yield richer datasets. They reduce the risk that testers will report erroneous effects and provide greater confidence in the test results. The following example used a multi-method approach in order to achieve the test's goal in a more rigorous and defensible way.
Read more>>
_______________________________________________________________________________________________________________________________________
Combining Information: Reliability for the Stryker Family of Vehicles
Reliability is an essential element in assessing the operational suitability of Department of Defense (DoD) weapon systems. It takes a prominent role in both the design and the analysis of operational tests. In the current era of reduced budgets and increased reliability requirements, it is challenging to verify reliability requirements in a single test. However, there has been an increased interest in capturing data consistently across all stages of testing, and these data can be intelligently combined using statistical methods.
Read more>>
