Session Title | Speaker | Type | Recording | Materials | Year |
---|---|---|---|---|---|
Keynote Thursday Lunchtime Keynote Speaker |
T. Charles Clancy Bradley Professor of Electrical and Computer Engineering Virginia Tech ![]() (bio)
Charles Clancy is the Bradley Professor of Electrical and Computer Engineering at Virginia Tech where he serves as the Executive Director of the Hume Center for National Security and Technology. Clancy leads a range of strategic programs at Virginia Tech related to security, including the Commonwealth Cyber Initiative. Prior to joining VT in 2010, Clancy was an engineering leader in the National Security Agency, leading research programs in digital communications and signal processing. He received his PhD from the University of Maryland, MS from University of Illinois, and BS from the Rose-Hulman Institute of Technology. He is co-author to over 200 peer-reviewed academic publications, six books, over twenty patents, and co-founder to five venture-backed startup companies. |
Keynote |
![]() | 2019 |
|
Keynote Wednesday Keynote Speaker III |
Timothy Dare Deputy Director, Developmental Test, Evaluation, and Prototyping SES OUSD(R&E) ![]() (bio)
Mr. Timothy S. Dare is the Deputy Director for Developmental Test, Evaluation and Prototyping (DD(DTEP)). As the DD(DTEP), he serves as the principal advisor on developmental test and evaluation (DT&E) to the Secretary of Defense, Under Secretary of Defense for Research and Engineering, and Director of Defense Research and Engineering for Advanced Capabilities. Mr. Dare is responsible for DT&E policy and guidance in support of the acquisition of major Department of Defense (DoD) systems, and providing advocacy, oversight, and guidance to the DT&E acquisition workforce. He informs policy and advances leading edge technologies through the development of advanced technology concepts, and developmental and operational prototypes. By working closely with interagency partners, academia, industry and governmental labs, he identifies, develops and demonstrates multi-domain technologies and concepts that address high-priority DoD, multi-Service, and Combatant Command warfighting needs. Prior to his appointment in December 2018, Mr. Dare was a Senior Program Manager for program management and capture at Lockheed Martin (LM) Space. In this role he was responsible for the capture and execution phases of multiple Intercontinental Ballistic Missile programs for Minuteman III, including a new airborne Nuclear Command and Control (NC2) development program. His major responsibilities included establishing program working environments at multiple locations, policies, processes, staffing, budget and technical baselines. Mr. Dare has extensive T&E and prototyping experience. As the Engineering Program Manager for the $1.8B Integrated Space C2 programs for NORAD/NORTHCOM systems at Cheyenne Mountain, Mr. Dare was the Integration and Test lead focusing on planning, executing, and evaluating the integration and test phases (developmental and operational T&E) for Missile Warning and Space Situational Awareness (SSA) systems. Mr. Dare has also been the Engineering Lead/Integration and Test lead on other systems such as the Hubble Space Telescope; international border control systems; artificial intelligence (AI) development systems (knowledge-based reasoning); Service-based networking systems for the UK Ministry of Defence; Army C2 systems; Space Fence C2; and foreign intelligence, surveillance, and reconnaissance systems. As part of the Department’s strategic defense portfolio, Mr. Dare led the development of advanced prototypes in SSA C2 (Space Fence), Information Assurance (Single Sign-on), AI systems, and was the sponsoring program manager for NC2 capability development. Mr. Dare is a graduate of Purdue University and is a member of both the Association for Computing Machinery and Program Management Institute. He has been recognized by the U.S. Air Force for his contributions supporting NORAD/NORTHCOM’s strategic defense missions, and the National Aeronautics and Space Administration for his contributions to the original Hubble Space Telescope program. Mr. Dare holds a U.S. Patent for Single Sign-on architectures. |
Keynote |
![]() | 2019 |
|
Keynote Wednesday Lunchtime Keynote Speaker |
Jared Freeman Chief Scientist of Aptima and Chair of the Human Systems Division National Defense Industry Association ![]() (bio)
Jared Freeman, Ph.D., is Chief Scientist of Aptima and Chair of the Human Systems Division of the National Defense Industry Association. His research and publications address measurement, assessment, and enhancement of human learning, cognition, and performance in technologically complex military environments. |
Keynote |
![]() | 2019 |
|
Keynote Welcoming & Opening Keynote-Tuesday AM |
Mike Gilmore Director DOT&E ![]() (bio)
Link to Bio unavail |
Keynote | Materials | 2016 |
|
Keynote Lunch with Keynote Leadership Perspective |
Jon Hollday NASA ![]() (bio)
Link to Bio unavail |
Keynote | 2016 |
||
Keynote STAT Engineering Keynote-Wednesday AM |
Christine Anderson Cook Statistics Los Alamos National Lab ![]() (bio)
s in the areas of complex system reliability, non-proliferation, malware detection and statistical process control. Before joining LANL, she was a faculty member in the Department of Statistics at Virginia Tech for 8 years. Her research areas include response surface methodology, design of experiments, reliability, multiple criterion optimization and graphical methods. She has authored more than 130 articles in statistics and quality peerreviewed journals, and has been a long time contributor to the Quality Progress Statistics Roundtable column. In 2012, she edited a special issue in Quality Engineering on Statistical Engineering with Lu Lu. She is an elected fellow of the American Statistical Association and the American Society for Quality. In 2012 she was honored with the ASQ Statistics Division William G. Hunter Award. In 2011 she received the 26th Annual Governor’s Award for Outsta |
Keynote | 2016 |
||
Keynote Lunch With Leadership Perspective-Wednesday PM |
David Brown Deputy Assistant Secretary of Defense, Developmental Test & Evaluation ![]() (bio)
Link to Bio unavail |
Keynote | 2016 |
||
Keynote Test and Evaluation Matter for Warfighter and We Need your Help to Make it Better |
Katherine Warner Science Advisor within Secretary of Defense DOT&E ![]() (bio)
Dr. Catherine Warner serves as the Science Advisor for the Director, Operational Test and Evaluation within the Office of the Secretary of Defense. Dr. Warner has been involved with operational test and evaluation since 1991, when she became a research staff member at the Institute for Defense Analyses (IDA). In that capacity, Dr. Warner performed and directed analysis of operational tests for Army, Navy, and Air Force systems in support of DOT&E. From 2005 – 2010, Dr. Warner was an Assistant Director at IDA and also served as the lead for the Air Warfare group. Her analysis portfolio included major aircraft systems such as the F-22, F/A-18E/F, V-22, and H-1. Prior to that, Dr. Warner was the lead analyst for Unmanned Aerial Vehicle (UAV) systems including Predator, Shadow, Hunter, and Global Hawk. In 2013 at the request of the Defense Information Systems Agency (DISA), Dr. Warner deployed to Kabul, Afghanistan for 16 months in support of NATO’s International Security Assistance Force (ISAF) and the US Operation Enduring Freedom (OEF). She lead a team of Information Technology specialists in advising Afghanistan’s Ministry of Communications and Information Technology on enhancing national communication capabilities for the security and economic growth of the country. The primary focus of this team included supporting the completion of Afghanistan’s National Fiber Optic Ring, Spectrum Management, and Cyber Security.Dr. Warner previously worked at the Lawrence Livermore National Laboratory. She grew up in Albuquerque, New Mexico, attended the University of New Mexico and San Jose State University where she earned both B.S. and M.S. degrees in Chemistry. She also earned both M.A. and Ph.D. degrees in Chemistry from Princeton University. |
Keynote | Materials | 2017 |
|
Keynote Retooling Design and Development |
Chris Singer NASA Deputy Chief Engineer NASA ![]() (bio)
Christopher (Chris) E. Singer is the NASA Deputy, Chief Engineer responsible for integrating engineering across the Agencies 10 field centers. Prior to this appointment in April 2016, he served as the Engineering Director at NASA’s Marshall Space Flight Center in Huntsville, Alabama. Appointed in 2011, Mr. Singer led an organization of 1,400 civil service and 1,200 support contractor employees responsible for the design, testing, evaluation, and operation of hardware and software associated with space transportation, spacecraft systems, science instruments and payloads under development at the Marshall Center. The Engineering Directorate also manages NASA’s Payload Operations Center at Marshall, which is the command post for scientific research activities on-board the International Space Station. Mr. Singer began his NASA career in 1983 as a rocket engine specialist. In 1992, he served a one-year assignment at NASA Headquarters in Washington, DC, as senior manager for the space shuttle main engine and external tank in the Space Shuttle Support Office. In 1994, Mr. Singer supervised the development and implementation of safety improvements and upgrades to shuttle propulsion components. In 2000, he was appointed chief engineer in the Space Transportation Directorate then was selected as deputy director of Marshall’s Engineering Directorate from 2004 to 2011. Mr. Singer is an AIAA Associate Fellow. In 2006, he received the Presidential Rank Award for Meritorious Executives — the highest honor for career federal employees. He was awarded the NASA Outstanding Leadership Medal in 2001 and 2008 for his leadership. In 1989, he received the prestigious Silver Snoopy Award from the Astronaut Corps for his contributions to the success of human spaceflight missions. A native of Nashville, Tennessee, Mr. Singer earned a bachelor’s degree in mechanical engineering in 1983 from Christian Brothers University in Memphis, Tennessee. Chris enjoys woodworking, fishing and Hang gliding. Chris is married to the former Jody Adams of Hartselle, Alabama. They have three children and live in Huntsville, Alabama. |
Keynote | Materials | 2017 |
|
Keynote Reflections on Statistical Engineering and it’s Application |
Geoff Vining Professor Virginia Tech ![]() (bio)
Geoff Vining is a Professor of Statistics at Virginia Tech. From 1999 – 2006, he also was the department head. He currently is the ASQ Treasurer for 2016 and Past-Chair of the ASQ Technical Communities Council. He is a Fellow of the ASQ, a Fellow of the American Statistical Association (ASA), and an Elected Member of the International Statistical Institute. Dr. Vining served as Editor of the Journal of Quality Technology from 1998 – 2000 and as Editor-in-Chief of Quality Engineering from 2008-2009. He also has served as Chair of the ASQ Publications Management Board, as Chair of the ASQ Statistics Division, and as Chair of the ASA Quality and Productivity Section. Dr. Vining won the 2010 Shewhart Medal, the ASQ career award given annually to the person not previously so honored who has demonstrated the most outstanding technical leadership in the field of modern quality control, especially through the development to its theory, principles, and techniques. He also received the 2015 Box Medal from the European Network for Business and Industrial Statistics (ENBIS). This medal recognizes each year an extraordinary statistician who has remarkably contributed with his/her work to the development and the application of statistical methods in European business and industry. In 2013, he received an Engineering Excellence Award from the NASA Engineering and Safety Center. He received the 2011 William G. Hunter Award from the ASQ Statistics Division for excellence in statistics as a communicator, a consultant, an educator, an innovator, an integrator of statistics with other disciplines and an implementer who obtains meaningful results. He won the 1990 ASQ Brumbaugh Award for the paper published in an ASQ journal that made the greatest contribution to the development of industrial applications of quality control and the 2005 Lloyd Nelson Award from the Statistics Division for the paper published in the Journal of Quality Technology that had the greatest immediate impact to practitioners. Dr. Vining is the author of three textbooks. He is an internationally recognized expert in the use of experimental design for quality, productivity, and reliability improvement and in the application of statistical process control. He has extensive consulting experience, most recently with NASA and the U.S. Department of Defense. |
Keynote | Materials | 2017 |
|
Keynote Developmental Test and Evaluation |
Brian Hall Principal Deputy Director DoD ![]() (bio)
Dr. Hall was appointed to the Senior Executive Service in November 2014 as the Principal Deputy Director for Developmental Test and Evaluation, and is currently serving as the Principle Deputy Director of the Test Resource Management Center (TRMC). In this position, he oversees matters concerning the Nation’s critical test range infrastructure, science and technology efforts, development of the biennial Strategic Plan for DoD Test and Evaluation (T&E) resources, as well as certification of the Services’ T&E budgets. Prior to this position, Dr. Hall was the Technical Advisor for Operational Test and Evaluation (OT&E) of all Land and Expeditionary Warfare systems in OSD-DOT&E. In this position, advised the highest authorities in DoD for OT&E, observed operational testing, and coauthored Operational Assessments and Beyond Low-Rate Initial Production reports submitted to the 4 Congressional Defense Committees. Prior to serving on the OSD staff, Dr. Hall was the Division Chief for Aviation, Missiles, and C4ISR Systems in the Army Test and Evaluation Command (ATEC), where he was responsible for supervising the development of test plans and evaluations that directly supported milestone decision reviews and materiel fielding/production decisions of more than 300 Army programs. While with the Army, Dr. Hall was one of the leading reliability experts that helped establish the Center for Reliability Growth at Aberdeen Proving Ground, as well as develop and administer the ATEC/AMSAA 3-day reliability course to improve defense acquisition learning. Over his career, Dr. Hall has led studies, developed methodologies, presented research, published papers, crafted policy, and authored policy implementation guides. He has also developed staff, advised numerous defense programs, and served as an executive member of, or invited contributor to: tri-service DoD Blue Ribbon Panels; National Academy of Science studies; and DoD working-groups to improve system reliability. Dr. Hall is a Senior Service College graduate and has earned advanced degrees in Applied Mathematics, Reliability Engineering, and Strategic Studies. He is a domain expert in Reliability Engineering, as well as in Reliability Growth Management and Methodology. He has developed statistical methods and reliability growth models that have been: published in international journals, incorporated into Military Handbooks, adopted by Operational Test Agency policy, and utilized to shape growth plans and assess reliability maturity of numerous DoD systems. |
Keynote | Materials | 2017 |
|
Keynote
|
Dave Duma Acting Director, Operationak Test and Evaluation, Office of the Secretary of Defense DOT&E ![]() (bio)
Mr. Duma is the Acting Director, Operational Test and Evaluation as of January 20, 2017. Mr. Duma was appointed as the Principal Deputy Director, Operational Test and Evaluation in January 2002. In this capacity he is responsible for all functional areas assigned to the office. He participates in the formulation, development, advocacy, and oversight of policies of the Secretary of Defense and in the development and implementation of test and test resource programs. He oversees the planning, conduct, analysis, evaluation, and reporting of operational and live fire testing. He serves as the Appropriation Director and Comptroller for the Operational Test and Evaluation, Defense Appropriation and coordinates all Planning, Programming, and Budgeting Execution matters. He previously served as Acting Director, Operational Test and Evaluation from February 2005 to July 2007 and again from May 2009 to September 2009. Mr. Duma also served as the Acting Deputy Director, Operational Test and Evaluation from January 1992 to June 1994. In this capacity he was responsible for oversight of the planning, conduct, analysis, and reporting of operational test and evaluation for all major conventional weapons systems in the Department of Defense. He supervised the development of evaluation plans and test program strategies, observed the conduct of operational test events, evaluated operational field tests of all armed services and submitted final reports for Congress. Mr. Duma returned to government service from the commercial sector. In private industry he worked a variety of projects involving test and evaluation; requirements generation; command, control, communications, intelligence, surveillance and reconnaissance; modeling and simulation; and software development. Mr. Duma has 30 years of naval experience during which he was designated as a Joint Service Officer. He served as the Director, Test and Evaluation Warfare Systems for the Chief of Naval Operations, the Deputy Commander, Submarine Squadron TEN, and he commanded the nuclear powered submarine USS SCAMP (SSN 588). Mr. Duma holds Masters of Science degrees in National Security and Strategic Studies and in Management. He holds a Bachelor of Science degree in Nuclear Engineering. He received the U.S. Presidential Executive Rank Award on two occasions; in 2008, the Meritorious Executive Award and in 2015, the Distinguished Executive Rank Award. He is a member of the International Test and Evaluation Association. |
Keynote | 2017 |
||
Keynote Opening Keynote |
David Chu President IDA ![]() (bio)
David Chu serves as President of the Institute for Defense Analyses. IDA is a non-profit corporation operating in the public interest. Its three federally funded research and development centers provide objective analyses of national security issues and related national challenges, particularly those requiring extraordinary scientific and technical expertise. As president, Dr. Chu directs the activities of more than 1,000 scientists and technologists. Together, they conduct and support research requested by federal agencies involved in advancing national security and advising on science and technology issues. Dr. Chu served in the Department of Defense as Under Secretary of Defense for Personnel and Readiness from 2001-2009, and earlier as Assistant Secretary of Defense and Director for Program Analysis and Evaluation from 1981-1993. From 1978-1981 he was the Assistant Director of the Congressional Budget Office for National Security and International Affairs. Dr. Chu served in the U. S. Army from 1968-1970. He was an economist with the RAND Corporation from 1970-1978, director of RAND’s Washington Office from 1994-1998, and vice president for its Army Research Division from 1998-2001. He earned a bachelor of arts in economics and mathematics, and his doctorate in economics, from Yale University. Dr. Chu is a member of the Defense Science Board and a Fellow of the National Academy of Public Administration. He is a recipient of the Department of Defense Medal for Distinguished Public Service with Gold Palm, the Department of Veterans Affairs Meritorious Service Award, the Department of the Army Distinguished Civilian Service Award, the Department of the Navy Distinguished Public Service Award, and the National Academy of Public Administration’s National Public Service Award. |
Keynote |
![]() | 2018 |
|
Keynote Closing Remarks |
Robert Behler Director DOT&E ![]() (bio)
Robert F. Behler was sworn in as Director of Operational Test and Evaluation on December 11, 2017. A Presidential appointee confirmed by the United States Senate, he serves as the senior advisor to the Secretary of Defense on operational and live fire test and evaluation of Department of Defense weapon systems. Prior to his appointment, he was the Chief Operating Officer and Deputy Director of the Carnegie Mellon University Software Engineering Institute (SEI), a Federally Funded Research and Development Center. SEI is a global leader in advancing software development and cybersecurity to solve the nation’s toughest problems through focused research, development, and transition to the broader software engineering community. Before joining the SEI, Mr. Behler was the President and CEO of SRC, Inc. (formerly the Syracuse Research Corporation). SRC is a not-for-profit research and development corporation with a forprofit manufacturing subsidiary that focuses on radar, electronic warfare and cybersecurity technologies. Prior to working at SRC, Mr. Behler was the General Manager and Senior Vice President of the MITRE Corp where he provided leadership to more than 2,500 technical staff in 65 worldwide locations. He joined MITRE from the Johns Hopkins University Applied Physics Laboratory where he was a General Manager for more than 350 scientists and engineers as they made significant contributions to critical Department of Defense (DOD) precision engagement challenges. General Behler served 31 years in the United States Air Force, retiring as a Major General in 2003. During his military career, he was the Principal Adviser for Command and Control, Intelligence, Surveillance and Reconnaissance (C21SR) to the Secretary and Chief of Staff of the U.S. Air Force (USAF). International assignments as a general officer included the Deputy Commander for NATO’s Joint Headquarters North in Stavanger, Norway. He was the Director of the Senate Liaison Office for the USAF during the 104th congress. Mr. Behler also served as the assistant for strategic systems to the Director of Operational Test and Evaluation. As an experimental test pilot, he flew more than 65 aircraft types. Operationally he flew worldwide reconnaissance missions in the fastest aircraft in the world, the SR-71 Blackbird. Mr. Behler is a Fellow of the Society of Experimental Test Pilots and an Associate Fellow of the American Institute of Aeronautics and Astronautics. He is a graduate of the University of Oklahoma where he received a B.S. and M.S. in aerospace engineering, has a MBA from Marymount University and was a National Security Fellow at the JFK School of Government at Harvard University. Mr. Behler has recently been on several National Research Council studies for the National Academy of Sciences including: “Critical Code,” “Software Producibility, Achieving Effective Acquisition of Information Technology in the Department of Defense” and “Development Planning: A Strategic Approach to Future Air Force Capabilities.” |
Keynote | 2018 |
||
Keynote Testing and Analytical Challenges on the Path to Hypersonic Flight |
Mark Lewis Director IDA-STPI ![]() (bio)
Dr. Mark J. Lewis is the Director of IDA’s Science and Technology Policy Institute, a federally funded research and development center. He leads an organization that provides analysis of national and international science and technology issues to the Office of Science and Technology Policy in the White House, as well as other Federal agencies including the National Institutes of Health, the National Science Foundation, NASA, the Department of Energy, Homeland Security, and the Federal Aviation Administration, among others.Prior to taking charge of STPI, Dr. Lewis served as the Willis Young, Jr. Professor and Chair of the Department of Aerospace Engineering at the University of Maryland. A faculty member at Maryland for 24 years, Dr. Lewis taught and conducted basic and applied research. From 2004 to 2008, Dr. Lewis was the Chief Scientist of the U.S. Air Force. From 2010 to 2011, he was President of the American Institute of Aeronautics and Astronautics (AIAA).Dr. Lewis attended the Massachusetts Institute of Technology, where he received a Bachelor of Science degree in aeronautics and astronautics, Bachelor of Science degree in earth and planetary science (1984), Master of Science (1985), and Doctor of Science (1988) in aeronautics and astronautics.Dr. Lewis is the author of more than 300 technical publications and has been an adviser to more than 70 graduate students. Dr. Lewis has also served on various advisory boards for NASA, the Air Force, and DoD, including two terms on the Air Force Scientific Advisory Board, the NASA Advisory Council, and the Aeronautics and Space Engineering Board of the National Academies.Dr. Lewis’s awards include the Meritorious Civilian Service Award and Exceptional Civilian Service Award; he was also recognized as the 1994 AIAA National Capital Young Scientist/Engineer of the Year, IECEC/ AIAA Lifetime Achievement Award, and is an Aviation Week and Space Technology Laureate (2007). |
Keynote |
![]() | 2018 |
|
Keynote Journey to a Data Centric Approcach for National Security |
Marcey Hoover Quality Assurance Director Sandia National Labortories ![]() (bio)
As Quality Assurance Director, Dr. Marcey Hoover is responsible for designing and sustaining the Laboratories’ quality assurance system and the associated technical capabilities needed for flawless execution of safe, secure, and efficient work to deliver exceptional products and services to its customers.Marcey previously served as the Senior Manager responsible for developing the science and engineering underpinning efforts to predict and influence the behavior of complex, highly interacting systems critical to our nation’s security posture. In her role as Senior Manager and Chief of Operations for Sandia’s Energy and Climate program, Marcey was responsible for strategic planning, financial management, business development, and communications. In prior positions, she managed organizations responsible for (1) quality engineering on new product development programs, (2) research and development of advanced computational techniques in the engineering sciences, and (3) development and execution of nuclear weapon testing and evaluation programs. Marcey has also led several executive office functions, including corporate- level strategic planning.Active in both the American Statistical Association and the American Society for Quality (ASQ), Marcey served two terms as the elected ASQ Statistics Division Treasurer. She was recognized as the Outstanding Alumni of the Purdue University Statistics Department in 2009 and nominated in 2011 for the YWCA Middle Rio Grande Women on the Move award. She currently serves on both the Purdue Strategic Research Advisory Council and the Statistics Alumni Advisory Board, and as a mentor for Big Brothers Big Sisters of New Mexico.Marcey received her bachelor of science degree in mathematics from Michigan State University, and her master of science and doctor of philosophy degrees in mathematical statistics from Purdue University. |
Keynote |
![]() | 2018 |
|
Keynote NASA AERONAUTICS |
Bob Pearce Deputy Associate Administrator Strategy, Aeronautics Research Mission DirectorateNASA ![]() (bio)
Mr. Pearce is responsible for leading aeronautics research mission strategic planning to guide the conduct of the agency’s aeronautics research and technology programs, as well as leading ARMD portfolio planning and assessments, mission directorate budget development and approval processes, and review and evaluation of all of NASA’s aeronautics research mission programs for strategic progress and relevance. Pearce is also currently acting director for ARMD’s Airspace Operations and Safety Program, and responsible for the overall planning, management and evaluation of foundational air traffic management and operational safety research. Previously he was director for strategy, architecture and analysis for ARMD, responsible for establishing a strategic systems analysis capability focused on understanding the system-level impacts of NASA’s programs, the potential for integrated solutions, and the development of high-leverage options for new investment and partnership. From 2003 until July 2010, Pearce was the deputy director of the FAA-led Next Generation Air Transportation System (NextGen) Joint Planning and Development Office (JPDO). The JPDO was an interagency office tasked with developing and facilitating the implementation of a national plan to transform the air transportation system to meet the long-term transportation needs of the nation. Prior to the JPDO, Pearce held various strategic and program management positions within NASA. In the mid-1990s he led the development of key national policy documents including the National Science and Technology Council’s “Goals for a National Partnership in Aeronautics Research and Technology” and the “Transportation Science and Technology Strategy.” These two documents provided a substantial basis for NASA’s expanded investment in aviation safety and airspace systems. He began his career as a design engineer at the Grumman Corporation, working on such projects as the Navy’s F-14 Tomcat fighter and DARPA’s X-29 Forward Swept Wing Demonstrator. Pearce also has experience from the Department of Transportation’s Volpe National Transportation Systems Center where he made contributions in the area of advanced concepts for intercity transportation systems. Pearce has received NASA’s Exceptional Service Medal for sustained excellence in planning and advocating innovative aeronautics programs in conjunction with the White House and other federal agencies. He received NASA’s Exceptional Achievement Medal for outstanding leadership of the JPDO in support of the transformation of the nation’s air transportation system. Pearce has also received NASA’s Cooperative External Achievement Award and several Exceptional Performance and Group Achievement Awards.He earned a bachelor’s of science degree in mechanical and aerospace engineering from Syracuse University, and a master’s of science degree in technology and policy from the Massachusetts Institute of Technology. |
Keynote | Materials | 2018 |
|
Keynote High-Effective Statistical Collaboration. The Art and the Science |
Peter Parker Team Lead Advance Measurement Systems NASA ![]() (bio)
Dr. Parker is Team Lead for Advanced Measurement Systemsat the National Aeronautics and Space Administration’s Langley Research Center in Hampton, Virginia. He serves an Agency-wide statistical expert across all of NASA’s mission directorates of Exploration, Aeronautics, and Science to infuse statistical thinking, engineering, and methods including statistical design of experiments, response surface methodology, and measurement system characterization. His expertise is in collaboratively integrating research objectives, measurement sciences, test design, and statistical methods to produce actionable knowledge for aerospace research and development. He holds a B.S. in Mechanical Engineering, a M.S. in Applied Physics and Computer Science and a M.S. and Ph.D. in Statistics from Virginia Tech. Dr. Parker is a senior member of the American Institute for Aeronautics and Astronautics, American Society for Quality, and the American Statistical Association. Dr. Parker currently Chairs the American Society for Quality’s Publication Management Board and previously served as Editor-in-Chief of the journal Quality Engineering. |
Keynote |
![]() | 2018 |
|
Keynote Consensus Building |
Antonio Possolo NIST Fellow, Chief Statistician National Institute of Standards and Technology. ![]() (bio)
Antonio Possolo holds a Ph.D. in statistics from Yale University, and has been practicing the statistical arts for more than 35 years, in industry (General Electric, Boeing), academia (Princeton University, University of Washington in Seattle, Classical University of Lisboa), and government. He is committed to the development and application of probabilistic and statistical methods that contribute to advances in science and technology, and in particular to measurement science. |
Keynote | Materials | 2018 |
|
Keynote Opening Keynote |
Dave Duma Assistant Director Operational Test and Evaluation ![]() (bio)
Mr. Duma is the Acting Director, Operational Test and Evaluation as of January 20, 2017. Mr. Duma was appointed as the Principal Deputy Director, Operational Test and Evaluation in January 2002. In this capacity he is responsible for all functional areas assigned to the office. He participates in the formulation, development, advocacy, and oversight of policies of the Secretary of Defense and in the development and implementation of test and test resource programs. He oversees the planning, conduct, analysis, evaluation, and reporting of operational and live fire testing. He serves as the Appropriation Director and Comptroller for the Operational Test and Evaluation, Defense Appropriation and coordinates all Planning, Programming, and Budgeting Execution matters. He previously served as Acting Director, Operational Test and Evaluation from February 2005 to July 2007 and again from May 2009 to September 2009.Mr. Duma also served as the Acting Deputy Director, Operational Test and Evaluation from January 1992 to June 1994. In this capacity he was responsible for oversight of the planning, conduct, analysis, and reporting of operational test and evaluation for all major conventional weapons systems in the Department of Defense. He supervised the development of evaluation plans and test program strategies, observed the conduct of operational test events, evaluated operational field tests of all armed services and submitted final reports for Congress.Mr. Duma returned to government service from the commercial sector. In private industry he worked a variety of projects involving test and evaluation; requirements generation; command, control, communications, intelligence, surveillance and reconnaissance; modeling and simulation; and software development.Mr. Duma has 30 years of naval experience during which he was designated as a Joint Service Officer. He served as the Director, Test and Evaluation Warfare Systems for the Chief of Naval Operations, the Deputy Commander, Submarine Squadron TEN, and he commanded the nuclear powered submarine USS SCAMP (SSN 588).Mr. Duma holds Masters of Science degrees in National Security and Strategic Studies and in Management. He holds a Bachelor of Science degree in Nuclear Engineering. He received the U.S. Presidential Executive Rank Award on two occasions; in 2008, the Meritorious Executive Award and in 2015, the Distinguished Executive Rank Award. He is a member of the International Test and Evaluation Association. |
Keynote |
![]() | 2018 |
|
Keynote Assessing Human-Autonomy Interaction in Driving-Assist Settings (Abstract)
In order to determine how the perception, Autopilot, and driver monitoring systems of Tesla Model 3s interact with one another, and also to determine the scale of between- and within-car variability, a series of four on-road tests were conducted. Three sets of tests were conducted on a closed track and one was conducted on a public highway. Results show wide variability across and within three Tesla Model 3s, with excellent performance in some cases but also likely catastrophic performance in others. This presentation will not only highlight how such interactions can be tested, but also how results can inform requirements and designs of future autonomous systems. |
Mary “Missy” Cummings Professor Duke University ![]() (bio)
Professor Mary (Missy) Cummings received her B.S. in Mathematics from the US Naval Academy in 1988, her M.S. in Space Systems Engineering from the Naval Postgraduate School in 1994, and her Ph.D. in Systems Engineering from the University of Virginia in 2004. A naval pilot from 1988-1999, she was one of the U.S. Navy’s first female fighter pilots. She is currently a Professor in the Duke University Electrical and Computer Engineering Department and the Director of the Humans and Autonomy Laboratory. She is an AIAA Fellow and a member of the Veoneer, Inc. Board of Directors |
Keynote | Session Recording |
![]() | 2021 |
Keynote Opening Remarks (Abstract)
Norton A. Schwartz serves as President of the Institute for Defense Analyses (IDA), a nonprofit corporation operating in the public interest. IDA manages three Federally Funded Research and Development Centers that answer the most challenging U.S. security and science policy questions with objective analysis leveraging extraordinary scientific, technical, and analytic expertise. At IDA, General Schwartz (U.S. Air Force, retired) directs the activities of more than 1,000 scientists and technologists employed by IDA. General Schwartz has a long and prestigious career of service and leadership that spans over 5 decades. He was most recently President and CEO of Business Executives for National Security (BENS). During his 6-year tenure at BENS, he was also a member of IDA’s Board of Trustees. Prior to retiring from the U.S. Air Force, General Schwartz served as the 19th Chief of Staff of the U.S. Air Force from 2008 to 2012. He previously held senior joint positions as Director of the Joint Staff and as the Commander of the U.S. Transportation Command. He began his service as a pilot with the airlift evacuation out of Vietnam in 1975. General Schwartz is a U.S. Air Force Academy graduate and holds a master’s degree in business administration from Central Michigan University. He is also an alumnus of the Armed Forces Staff College and the National War College. He is a member of the Council on Foreign Relations and a 1994 Fellow of Massachusetts Institute of Technology’s Seminar XXI. General Schwartz has been married to Suzie since 1981. |
Norton Schwartz President Institute for Defense Analyses ![]() |
Keynote | Session Recording | 2021 |
|
Keynote Closing Remarks (Abstract)
Mr. William (Allen) Kilgore serves as Director, Research Directorate at NASA Langley Research Center. He previously served as Deputy Director of Aerosciences providing executive leadership and oversight for the Center’s Aerosciences fundamental and applied research and technology capabilities with the responsibility over Aeroscience experimental and computational research. After being appointed to the Senior Executive Service (SES) in 2013, Mr. Kilgore served as the Deputy Director, Facilities and Laboratory Operations in the Research Directorate. Prior to this position, Mr. Kilgore spent over twenty years in the operations of NASA Langley’s major aerospace research facilities including budget formulation and execution, maintenance, strategic investments, workforce planning and development, facility advocacy, and integration of facilities’ schedules. During his time at Langley, he has worked in nearly all of the major wind tunnels with a primary focus on process controls, operations and testing techniques supporting aerosciences research. For several years, Mr. Kilgore led the National Transonic Facility, the world’s largest cryogenic wind tunnel. Mr. Kilgore has been at NASA Langley Research Center since 1989, starting as a graduate student. Mr. Kilgore earned a B.S. and M.S. in Mechanical Engineering with concentration in dynamics and controls from Old Dominion University in 1984 and 1989, respectively. He is the recipient of NASA’s Exceptional Engineering Achievement Medal in 2008 and Exceptional Service Medal in 2012. |
William “Allen” Kilgore Director, Research Directorate NASA Langley Research Center ![]() |
Keynote | Session Recording | 2021 |
|
Keynote Opening Remarks (Abstract)
Dr. O’Toole is the Acting Director, Operational Test and Evaluation as of January 20, 2021. Dr. O’Toole was appointed as the Principal Deputy Director, Operational Test and Evaluation in February 2020. In this capacity he is the principal staff assistant for all functional areas assigned to the office. He participates in the formulation, development, advocacy, and oversight of policies of the Secretary of Defense and in the development and implementation of test and test resource programs. He supports the Director in the planning, conduct, evaluation and reporting of operational and live fire testing. He serves as the Appropriation Director and Comptroller for the Operational Test and Evaluation, Defense Appropriation and the principal advisor to the Director on all Planning, Programming, and Budgeting System matters. Dr. O’Toole is the former Deputy Director for Naval Warfare within DOT&E. He oversaw the operational and live-fire testing of ships and submarines and their associated sensors; combat and communications systems, and weapons. He was also responsible for overseeing the adequacy of the test infrastructure and resources to support operational and live-fire testing for all acquisition programs across the Defense Department. Dr. O’Toole was previously an employee of the Naval Sea Systems Command as the Deputy Group Director of Aircraft Carrier Design and Systems Engineering. Prior to that, he was the Director of Systems Engineering Division (Submarines and Undersea Systems) where he led a diverse team of engineers who supported all Submarine Program Managers. His other assignments include being a Ship Design Manager/Navy’s Technical Authority for the USS VIRGINIA Class submarines during design and new construction and for Amphibious Ships, Auxiliary Ships, and Command & Control Ships during inservice operations. Dr. O’Toole has also held other positions within the Department of Defense such as Deputy Program Executive Officer (Maritime and Rotary Wing) at the United States Special Operations Acquisition Command, Staff to the Deputy Assistant Secretary of the Navy for Research, Development & Acquisition (Ship Programs), and Deputy Director of Regional Maintenance for COMPACFLT (N43). In addition, Dr. O’Toole has over 30 years of experience as a Naval Officer (Active and Reserve) retiring at the rank of CAPTAIN. His significant tours include 5 Commanding Officer tours. Dr. Raymond D. O’Toole, Jr. is a native of Long Island NY and a graduate of the State University of New York – Maritime College earning a Bachelor of Engineering in Marine Engineering. He also holds a Master of Engineering Degree in Systems Engineering from Virginia Polytechnic Institute and State University, a Master of Science Degree in National Resource Strategy from the Industrial College of the Armed Forces, and a Doctorate in Engineering in the field of Engineering Management from the George Washington University, where he is now a Professional Lecturer of Engineering Management and Systems Engineering. He has received the SECDEF Meritorious Civilian Service Award and the USN Meritorious and Superior Civilian Service Awards. |
Raymond O’Toole Acting Director, Operational Test and Evaluation DOT&E ![]() |
Keynote | Session Recording | 2021 |
|
Keynote April 28 Opening Keynote |
Nickolas Guertin Director, Operational Test & Evaluation, OSD/DOT&E ![]() (bio)
Nickolas H. Guertin was sworn in as Director, Operational Test and Evaluation on December 20, 2021. A Presidential appointee confirmed by the United States Senate, he serves as the senior advisor to the Secretary of Defense on operational and live fire test and evaluation of Department of Defense weapon systems. Mr. Guertin has an extensive four-decade combined military and civilian career in submarine operations, ship construction and maintenance, development and testing of weapons, sensors, combat management products including the improvement of systems engineering, and defense acquisition. Most recently, he has performed applied research for government and academia in software-reliant and cyber-physical systems at Carnegie Mellon University’s Software Engineering Institute. Over his career, he has been in leadership of organizational transformation, improving competition, application of modular open system approaches, as well as prototyping and experimentation. He has also researched and published extensively on software-reliant system design, testing and acquisition. He received a BS in Mechanical Engineering from the University of Washington and an MBA from Bryant University. He is a retired Navy Reserve Engineering Duty Officer, was Defense Acquisition Workforce Improvement Act (DAWIA) certified in Program Management and Engineering, and is also a registered Professional Engineer (Mechanical). Mr. Guertin is involved with his community as an Assistant Scoutmaster and Merit Badge Counselor for two local Scouts BSA troops as well as being an avid amateur musician. He is a native of Connecticut and now resides in Virginia with his wife and twin children. |
Keynote | Session Recording | 2022 |
Session Title | Speaker | Type | Recording | Materials | Year |
---|---|---|---|---|---|
Keynote Thursday Lunchtime Keynote Speaker |
T. Charles Clancy Bradley Professor of Electrical and Computer Engineering Virginia Tech ![]() |
Keynote |
![]() | 2019 |
|
Keynote Wednesday Keynote Speaker III |
Timothy Dare Deputy Director, Developmental Test, Evaluation, and Prototyping SES OUSD(R&E) ![]() |
Keynote |
![]() | 2019 |
|
Keynote Wednesday Lunchtime Keynote Speaker |
Jared Freeman Chief Scientist of Aptima and Chair of the Human Systems Division National Defense Industry Association ![]() |
Keynote |
![]() | 2019 |
|
Keynote Welcoming & Opening Keynote-Tuesday AM |
Mike Gilmore Director DOT&E ![]() |
Keynote | Materials | 2016 |
|
Keynote Lunch with Keynote Leadership Perspective |
Jon Hollday NASA ![]() |
Keynote | 2016 |
||
Keynote STAT Engineering Keynote-Wednesday AM |
Christine Anderson Cook Statistics Los Alamos National Lab ![]() |
Keynote | 2016 |
||
Keynote Lunch With Leadership Perspective-Wednesday PM |
David Brown Deputy Assistant Secretary of Defense, Developmental Test & Evaluation ![]() |
Keynote | 2016 |
||
Keynote Test and Evaluation Matter for Warfighter and We Need your Help to Make it Better |
Katherine Warner Science Advisor within Secretary of Defense DOT&E ![]() |
Keynote | Materials | 2017 |
|
Keynote Retooling Design and Development |
Chris Singer NASA Deputy Chief Engineer NASA ![]() |
Keynote | Materials | 2017 |
|
Keynote Reflections on Statistical Engineering and it’s Application |
Geoff Vining Professor Virginia Tech ![]() |
Keynote | Materials | 2017 |
|
Keynote Developmental Test and Evaluation |
Brian Hall Principal Deputy Director DoD ![]() |
Keynote | Materials | 2017 |
|
Keynote
|
Dave Duma Acting Director, Operationak Test and Evaluation, Office of the Secretary of Defense DOT&E ![]() |
Keynote | 2017 |
||
Keynote Opening Keynote |
David Chu President IDA ![]() |
Keynote |
![]() | 2018 |
|
Keynote Closing Remarks |
Robert Behler Director DOT&E ![]() |
Keynote | 2018 |
||
Keynote Testing and Analytical Challenges on the Path to Hypersonic Flight |
Mark Lewis Director IDA-STPI ![]() |
Keynote |
![]() | 2018 |
|
Keynote Journey to a Data Centric Approcach for National Security |
Marcey Hoover Quality Assurance Director Sandia National Labortories ![]() |
Keynote |
![]() | 2018 |
|
Keynote NASA AERONAUTICS |
Bob Pearce Deputy Associate Administrator Strategy, Aeronautics Research Mission DirectorateNASA ![]() |
Keynote | Materials | 2018 |
|
Keynote High-Effective Statistical Collaboration. The Art and the Science |
Peter Parker Team Lead Advance Measurement Systems NASA ![]() |
Keynote |
![]() | 2018 |
|
Keynote Consensus Building |
Antonio Possolo NIST Fellow, Chief Statistician National Institute of Standards and Technology. ![]() |
Keynote | Materials | 2018 |
|
Keynote Opening Keynote |
Dave Duma Assistant Director Operational Test and Evaluation ![]() |
Keynote |
![]() | 2018 |
|
Keynote Assessing Human-Autonomy Interaction in Driving-Assist Settings |
Mary “Missy” Cummings Professor Duke University ![]() |
Keynote | Session Recording |
![]() | 2021 |
Keynote Opening Remarks |
Norton Schwartz President Institute for Defense Analyses ![]() |
Keynote | Session Recording | 2021 |
|
Keynote Closing Remarks |
William “Allen” Kilgore Director, Research Directorate NASA Langley Research Center ![]() |
Keynote | Session Recording | 2021 |
|
Keynote Opening Remarks |
Raymond O’Toole Acting Director, Operational Test and Evaluation DOT&E ![]() |
Keynote | Session Recording | 2021 |
|
Keynote April 28 Opening Keynote |
Nickolas Guertin Director, Operational Test & Evaluation, OSD/DOT&E ![]() |
Keynote | Session Recording | 2022 |