Model Selection
After carefully planning and executing a test as reviewed in the Plan, Design, and Execute sections, the test should produce information-rich results capable of answering the questions that the test team initially set out to answer. However, this important information is often obscured by the shear amount of data. Moreover, initial impressions of test results can be misleading. Statistical analyses provide the tools necessary to making defensible statements about system characteristics by objectively summarizing the data, determining which factors are significant, and discovering other patterns in the data. Here, you can learn about best practices throughout the analytical process as well as the core concepts underlying statistical data analysis.
Analysis Checklist
Final analyses are impacted by every step of the test planning, data collection, and analysis preparation process, and it pays to document even nuanced actions that influence the data. Firm conclusions can only be made in the context of test and data history, and the test team must record and pass on this history as it occurs. Following a consistent approach to new data can help the test team appreciate important details, anticipate consequential actions, and maximize the rigor of their analyses. The analysis checklist provides a starting point and includes common steps that test teams can adopt or adapt for their specific needs. The sections below and linked within this checklist describe the stages in further detail and provide additional resources and best practices for each. Read more>>
Pre-Analysis
Transitioning from test execution and data collection to data analysis is no trivial task. Seemingly small errors in data entry or multiple methods of data reduction, for example, have the potential to render analyses inconclusive or lead to a anomalous result if they are not brought to the attention of the analysts. Furthermore, it can pay to organize and document datasets in a format that can easily be shared and explained to others. This section outlines steps to producing a usable dataset and highlights considerations that can save time and minimize frustrations throughout the analysis process.
Read more>>
Exploratory Data Analysis
Even with careful and detailed planning, it is always important to assess the nature and quality of the resulting data. Exploratory Data Analysis (EDA) is an approach to learning about a data set. The goal is to examine and summarize the data in order to make sense out of the otherwise overwhelming mass of information. EDA can help evaluators understand their data, find preliminary results, and determine an analysis approach or efficiently communicate analytical needs to statisticians.
Read more>>
Inferential Analysis
Inferential statistical analysis involves objectively and quantitatively summarizing the data, determining which data patterns are significant, and making inferential statements about system performance. Inferential statistics provide the tools necessary to identify critical factors and to what degree specific test results can be generalized to the system as a whole. Here, you can learn about the statistical foundations that allow evaluators to make more broad and stronger claims about systems than allowed by descriptive and exploratory analyses.
Read more>>
Data Processing & Analysis Tools
This section will include useful applications such as interactive Shiny.io apps for conducting simple tests, R code for more complicated procedures, and various other statistics resources. Proceed to Tools>>
