Curriculum

What is Test Science?

Statistical methods including design of experiments and the corresponding statistical data analysis are the core methodologies in a scientific approach to test planning and evaluation. Design of experiments elicits maximum information from constrained resources, provides a structured approach for leveraging information across multiple test events, and provides a defensible rationale for test adequacy and test size. Statistical analysis methods maximize knowledge gained from the testing, provide objective summaries of test data, and quantify uncertainty in the analysis. Statistical design and analysis methods support credible decision making by clearly quantifying where systems work.

We have pulled together a range of content, examples, and applications to guide every user through the process of designing, executing, analyzing, and evaluating a test.

The Test Science Curriculum provides a step-by-step process of designing, executing, and analyzing a test or experiment. At each step of the process, it is useful to consider if what is being done or thought could be improved and if that improvement will lead to a better test. Therefore, two sections recommending best practices are provided: one for planning and designing a test, and one for analyzing and evaluating the test. 

To practice the material, look to the Interactive Tools. Shiny applications, Excel spreadsheet calculators, and PDF diagrams are included in order to demonstrate and provide context to the content in the curriculum.

The Training and Resources section has training videos and webinars, case studies, and further research with concrete examples.

Step One: Plan

Set Test Goals 

Performance

Human Factors

Reliability

Select Response Variables   

Empirically Vetted

Custom

Select Factors  

Strategically Vary

Hold Constant

Record

 


Step Two: Design

Construct Test Design 

Design of Experiments

Common Designs

Design Choice

Assess Test Adequacy 

Confidence

Power

Collinearity & VIF

Scaled Prediction Variance

Execute the Test  

Best Practices for Execution

 

 

 


Step Three: Analysis

 Pre-Analysis

Create Dataset

Document Process

Clarify

 Data Analysis 

Visualize

Process & Clean

Summarize

Explore Multivariate Data

Inferential Analysis

Model Selection

 

 

 


Step Four: Evaluate

 Interpret 

Confidence Intervals

P-values

 

Draw Conclusions  

Summarize Results

Future Work

 

 


 

 


 

 


Leave a Reply

Your email address will not be published. Required fields are marked *