# Tools and Applications

Our tools come in many forms, ranging from web-based software applications to Excel spreadsheets and PowerPoint presentations, among other types of materials. You can search for the resources featured in the Planning, Designing, and Analyzing sections, or browse additional resources that evaluators commonly consult. Bayesian Binomial Credible Intervals
This app computes the Bayesian posterior central interval for binomially distributed data (e.g., hit/miss, success/fail outcomes). We offer three approaches for computing the posterior central interval. Two approaches are based on the use of objective, or noninformative priors. A third approach allows the user to enter a subjective prior based on historical data.Analysis
• Confidence Intervals Availability Exponential Confidence Intervals
Calculator (.XLSX)
This tool calculates an 80% confidence interval for operational availability assuming both uptimes and downtimes are exponentially distributed.Analysis
• Confidence Intervals Binomial Confidence Intervals
Calculates multiple confidence intervals for binomial (pass/fail) data.Analysis
• Confidence Intervals Lognormal Confidence Intervals
Calculates Wald Confidence Intervals about the mean, median, and percentiles for a single sample where the data best follow a lognormal distribution. Analysis
• Confidence Intervals
Bootstrap Estimates
Performs a low sample size bootstrap analysis (1000s). It provides means, medians, and standard deviations resulting from bootstrapping of a test data sample.Analysis Two Sample Kolmogorov-Smirnov Test
This application computes the Kolmogorov-Smirnov test statistic for comparing two distributions, provides report-ready visualizations of the probability density and empirical cumulative density functions, and generates a paragraph explaining the meaning of the test statistic and its associated p-value.Analysis
Lognormal Confidence Intervals
Calculates Wald Confidence Intervals about the mean and percentiles for a single sample analysis where the data best follow a lognormal distribution.Analysis
• Confidence Intervals
Duration Based Operating Characteristic Curves
Generates OC curves for exponential data after calculating maximum allowed failures given inputs of test length, reliability requirement, and desired level of confidence.Planning
• Reliability
Binomial Operating Characteristic Curves
Generates OC curves for binary reliability. Given consumer risk (beta), the maximum acceptable proportion of failures (requirement), and number of trials, this spreadsheet calculates the maximum number of failures which still results in acceptance of the system. Planning
• Reliability Parametric Reliability Models
This application fits three statistical distributions to censored or un-censored univariate reliability data and plots the failure probability, reliability, hazard, and probability density functions for each model. It provides comparison criteria for model selection and allows the user to download high-resolution images of the model graphs. The application also accommodates users who merely want to plot graphs based on existing parameter estimates from previously fitted reliability models.Analysis
NASA-TLX Scoring Sheet
Calculates raw and weighted individual and sample-level workload scores given raw responses to the NASA-TLX questionnaire.Analysis
• Survey Results
• Published Surveys Comparing Reliability Tests
This tool generates plots comparing the likelihood of different outcomes for reliability tests. It is applicable for any continuous measure of reliability, including time (e.g., hours) and distance (e.g., miles). It can also be used with discrete measures of reliability (e.g., rounds fired between failure) provided the numbers are sufficiently large. (Hundreds or thousands between failure are probably sufficient.)Design
• Analysis
System Usability Scale (SUS) Scoring Sheet
Coming Soon: Calculator (.XLSX)
Calculates overall usability scores given raw responses to the SUS questionnaire.Analysis
• Survey Results
• Published Surveys
Distribution Visualization
Plots probability density functions of distributions with user-adjustable parameters to aid comparison and understanding of "shape".Analysis One Sample t-test Power
Estimates power and plots power by sample size for a one sample t-test.Design
• Power Two Sample t-test Power
Estimates power and plots power by sample size for a two sample t-test.Design
• Power One Sample Proportion Test Power
Estimates power and plots power by sample size for a one sample proportion test.Design
• Power Categorical Analysis Power
Estimates power and plots power by sample size for a multi-factor categorical test.Design
• Power JEDIS JMP AddIn
JEDIS is an add-in for the JMP statistical software program that helps automate the Design of Experiments (DOE) process within JMP in a user-friendly manner. JEDIS builds multiple test designs in JMP over user-specified ranges of sample sizes, Signal-to-Noise Ratios (SNR), and alpha (1-confidence) levels. It then automatically calculates the statistical power to detect an effect due to each factor and any specified interactions for each design. When finished, JEDIS presents the statistical power vs. design metrics in interactive plots and stores the data in an easy to use format. Design
• Power GLM Power for Categorical Factors
Approximates power for effects in a generalized linear model for experiments with categorical factors. Currently supports Logit, Poisson, and linear regression.Design
• Power
Reliability Confidence Intervals
Calculates exact confidence intervals for the mean time between failures when failures follow an exponential distribution.Analysis
• Reliability
• Confidence Intervals
Fishbone Diagram
Documents initial list and sorting of factors that could affect test outcomes.Planning
• Factor Management
Input-Process-Output Diagram
Groups factors according to management strategy to assist in down-selection and design.Planning
• Factor Management
System Usability Scale (SUS) Documentation
Documents development of the SUS and provides questions, scoring and administration guidelines.Planning
• Survey
• Published Surveys
General Survey Questions
Provides generic wording for common questions in test and evaluation that can be tailored to specific systems.Planning
• Survey
• Custom Surveys
Provides instructions for administering and scoring the NASA-TLX. Originally hosted at NASA's TLX web archive. Planning
• Survey
• Published Surveys
NASA-TLX Survey Questions
Includes original NASA-TLX question wording, rating scale, and word-pairs for optional choice task. Planning
• Survey
• Published Surveys
Survey Design Best Practices Checklist
Lists best practices for writing and formatting custom surveys.Planning
• Survey
• Custom Surveys Power in JMP Tutorial
Provides an overview of power calculations and detailed
instructions for calculating power for Designed Experiments across a variety of software packages.
Design
• Power Top N Pareto Front Search
JMP Add-In, video demonstration of installation and use, and introduction document and video lecture on Top N Pareto Front Search.Analysis
• Solution Reduction Survey Exploratory Data Analysis
This web application analyzes Likert-scale survey data. Users upload .csv-formatted data tables, and the application produces customizable tables of statistics and plots of survey responses. Tables and plots can be downloaded for reporting or further analysis.Analysis
• Survey Results Operating Characteristic Curve for Acceptance Sampling by Attribute
This application provides an easy-to-use interface to compare resistance-to-penetration test plans (and other discrete test plans). Specifically, the application generates operating characteristic curves for single or double sampling test plans, and highlights the risk for each sampling plan. The application includes multiple distribution options, plotting capabilities and customizations, and downloadable graphics and data.Planning
• Reliability