What is Test Science?

Statistical methods including design of experiments and the corresponding statistical data analysis are the core methodologies in a scientific approach to test planning and evaluation. Design of experiments elicits maximum information from constrained resources, provides a structured approach for leveraging information across multiple test events, and provides a defensible rationale for test adequacy and test size. Statistical analysis methods maximize knowledge gained from the testing, provide objective summaries of test data, and quantify uncertainty in the analysis. Statistical design and analysis methods support credible decision making by clearly quantifying where systems work.

We have pulled together a range of content, examples, and applications to guide every user through the process of designing, executing, analyzing, and evaluating a test.

The Test Science Curriculum provides a step-by-step process of designing, executing, and analyzing a test or experiment. At each step of the process, it is useful to consider if what is being done or thought could be improved and if that improvement will lead to a better test. Therefore, two sections recommending best practices are provided: one for planning and designing a test, and one for analyzing and evaluating the test. 

To practice the material, look to the Interactive Tools. Shiny applications, Excel spreadsheet calculators, and PDF diagrams are included in order to demonstrate and provide context to the content in the curriculum.

The Training and Resources section has training videos and webinars, case studies, and further research with concrete examples.

Step One: Plan

Set Test Goals 


Human Factors


Select Response Variables   

Empirically Vetted


Select Factors  

Strategically Vary

Hold Constant



Step Two: Design

Construct Test Design 

Design of Experiments

Common Designs

Design Choice

Assess Test Adequacy 



Collinearity & VIF

Scaled Prediction Variance

Execute the Test  

Best Practices for Execution




Step Three: Analysis


Create Dataset

Document Process


 Data Analysis 


Process & Clean


Explore Multivariate Data

Inferential Analysis

Model Selection




Step Four: Evaluate


Confidence Intervals



Draw Conclusions  

Summarize Results

Future Work







Leave a Reply