DATAWorks Sessions Archive

In an experiment with a single factor with three levels, treatments A, B, and C, a single treatment is to be applied to each of several experimental units selected from some set of units. The response variable is continuous, and differences in its value show the relative effectiveness of the treatments. An experimental design will dictate which treatment is applied to what units. Since differences in the response variable are used to judge differences between treatments, the most important goal of the design is to prevent the treatment effect being masked by some unrelated property of the experimental units. Second important function of the design is to ensure power, that is, that if the treatments are not equally effective, the differences in the response variable are likely to be larger than background noise. Classical experimental design theory uses three principles: replication, randomization, and blocking, to produce an experimental design. Replication refers to how many units are used, blocking is a possible grouping of the units to reduce between unit heterogeneity, and randomization governs the assignment of units to treatment. Classical experimental designs are balanced as much as possible, that is, the three treatments are applied the same number of times, in each potential block of units. Bayesian experimental design aims to make use of additional related information, often called prior information, to produce a design. The information may be in the form of related experimental results, for example, treatments A and B may have been previously studied. It could be additional information about the experimental units, or about the response variable. This additional information could be used to change the usual blocking, to reduce the number of units assigned to treatments A and B compared to C, and/or reduce the total number of units needed to ensure power. This talk aims to explain Bayesian design concepts and illustrate them on realistic examples.

For chemical/biological testing, test chambers are sometimes designed with a vapor or aerosol homogeneity requirement. For example, a test community may require that the difference in concentration between any two test locations in a chamber be no greater than 20 percent. To validate the chamber, testers must demonstrate that such a requirement is met with a specified amount of certainty, such as 80 percent. With a validated chamber, multiple systems can be simultaneously tested at different test locations with the assurance that each system is exposed to nearly the same concentration. In some cases, however, homogeneity requirements are difficult to achieve. This presentation demonstrates a valid Bayesian method for testing probability of detection as a function of concentration in a chamber that fails to meet a homogeneity requirement. The demonstrated method of analysis is based on recent experience with an actual test chamber. Multiple systems are tested simultaneously at different locations in the chamber. Because systems tested in the chamber are exposed to different concentrations depending on these locations, the differences must be quantified to the greatest extent possible. To this end, data from the failed validation efforts are used to specify informative prior distributions for probability-of-detection modeling. Because these priors quantify and incorporate uncertainty in model parameters, they ensure that the final probability-of-detection model constitutes a valid comparison of the performance of the different systems.

Principal Component Analysis is a standard tool in an analyst’s toolbox. The standard practice of rescaling each column can be reframed as a copula-based decomposition in which the marginal distributions are fit with a univariate Gaussian distribution and the joint distribution is modeled with a Gaussian copula. In this light, we present an alternative to traditional PCA we call XPCA by relaxing the marginal Gaussian assumption and instead fit each marginal distribution with the empirical distribution function. Interval-censoring methods are used to account for the discrete nature of the empirical distribution function when fitting the Gaussian copula model. In this talk, we derive the XPCA estimator and inspect the differences in fits on both simulated and real data applications.

The average and standard deviation of, say, strength or dimensional test data are basic engineering math, simple to calculate. What those resulting values actually mean, however, may not be simple, and can be surprisingly different from what a researcher wants to calculate and communicate. Mistakes can lead to overlarge estimates of spread, structures that are over- or under-designed and other challenges to understanding or communicating what your data is really telling you. This talk will discuss some common errors and missed opportunities seen in engineering and scientific analyses along with mitigations that can be applied through smart and efficient test planning and analysis. It will cover when – and when not – to report a simple mean of a dataset based on the way the data was taken; why ignoring this often either hides or overstates risk; and a standard method for planning tests and analyses to avoid this problem. And it will cover what investigators can correctly (or incorrectly) say about means and standard deviations of data, including how and why to describe uncertainty and assumptions depending on what a value will be used for. The presentation is geared toward the engineer, scientist or project manager charged with test planning, data analysis or understanding findings from tests and other analyses. Attenders’ basic understanding of quantitative data analysis is recommended; more-experienced participants will grasp correspondingly more nuance from the pitch. Some knowledge of statistics is helpful, but not required. Participants will be challenged to think about an average as not just “the average”, but a valuable number that can and must relate to the engineering problem to be solved, and must be firmly based in the data. Attenders will leave the talk with a more sophisticated understanding of this basic, ubiquitous but surprisingly nuanced statistic and greater appreciation of its power as an engineering tool.