Session Title | Speaker | Type | Materials | Year |
---|---|---|---|---|
Panel Finding the Human in the Loop: Considerations for AI in Decision Making |
Joe Lyons Lead for the Collaborative Interfaces and Teaming Core Research Area 711 Human Performance Wing at Wright-Patterson AFB ![]() (bio)
Joseph B. Lyons is the Lead for the Collaborative Interfaces and Teaming Core Research Area within the 711 Human Performance Wing at Wright-Patterson AFB, OH. Dr. Lyons received his PhD in Industrial/Organizational Psychology from Wright State University in Dayton, OH, in 2005. Some of Dr. Lyons’ research interests include human-machine trust, interpersonal trust, human factors, and influence. Dr. Lyons has worked for the Air Force Research Laboratory as a civilian researcher since 2005, and between 2011-2013 he served as the Program Officer at the Air Force Office of Scientific Research where he created a basic research portfolio to study both interpersonal and human-machine trust as well as social influence. Dr. Lyons has published in a variety of peer-reviewed journals, and is an Associate Editor for the journal Military Psychology. Dr. Lyons is a Fellow of the American Psychological Association and the Society for Military Psychologists. |
Panel |
![]() Recording | 2021 |
Panel Finding the Human in the Loop: Panelist |
Chad Bieber Director, Test and Evaluation. Senior Research Engineer. Johns Hopkins University Applied Physics Laboratory ![]() (bio)
Chad Bieber is a Senior Research Engineer at the Johns Hopkins University Applied Physics Lab, is currently working as the Test and Evaluation Director for Project Maven, and was previously a Research Staff Member at IDA. A former pilot in the US Air Force, he received his Ph.D. in Aerospace Engineering from North Carolina State University. Chad is interested in how humans interact with complex, and increasingly autonomous, systems. |
Panel |
Recording | 2021 |
Panel Finding the Human in the Loop: Panelist |
Poornima Madhavan Principal Scientist and Capability Lead for Social and Behavioral Sciences MITRE ![]() (bio)
Dr. Poornima Madhavan is a Principal Scientist and Capability Lead for Social and Behavioral Sciences at the MITRE Corporation. She has more than 15 years of experience studying human-systems integration issues in sociotechnical systems including trust calibration, decision making, and risk perception. Dr. Madhavan spent the first decade of her career as a professor of Human Factors Psychology at Old Dominion University where she studied threat detection, risk analysis and human decision making in aviation and border security. This was followed by a stint as the Director of the Board on Human-Systems Integration at the National Academies of Sciences, Engineering and Medicine where she served as the primary spokesperson to the federal government on policy issues related to human-systems integration. Just before joining MITRE, Dr. Madhavan’s work focused on modeling human behavioral effects of non-lethal weapons and human-machine teaming for autonomous systems at the Institute for Defense Analyses. Dr. Madhavan received her M.A. and Ph.D. in Engineering Psychology from the University of Illinois at Urbana-Champaign and completed her post-doctoral fellowship in Social and Decision Sciences at Carnegie Mellon University. |
Panel |
Recording | 2021 |
Keynote Wednesday Keynote Speaker I |
Peter Parker Team Lead, Advanced Measurement Systems NASA ![]() (bio)
Dr. Parker is Team Lead for Advanced Measurement Systems at the National Aeronautics and Space Administration’s Langley Research Center in Hampton, Virginia. He serves an Agency-wide statistical expert across all of NASA’s mission directorates of Exploration, Aeronautics, and Science to infuse statistical thinking, engineering, and methods including statistical design of experiments, response surface methodology, and measurement system characterization. His expertise is in collaboratively integrating research objectives, measurement sciences, test design, and statistical methods to produce actionable knowledge for aerospace research and development. He holds a B.S. in Mechanical Engineering, a M.S. in Applied Physics and Computer Science and a M.S. and Ph.D. in Statistics from Virginia Tech. Dr. Parker is a senior member of the American Institute for Aeronautics and Astronautics, American Society for Quality, and the American Statistical Association. Dr. Parker currently Chairs the American Society for Quality’s Publication Management Board and previously served as Editor-in-Chief of the journal Quality Engineering. |
Keynote |
![]() | 2019 |
Keynote Tuesday Keynote |
David Chu President Institute for Defense Analyses ![]() (bio)
David Chu serves as President of the Institute for Defense Analyses. IDA is a non-profit corporation operating in the public interest. Its three federally funded research and development centers provide objective analyses of national security issues and related national challenges, particularly those requiring extraordinary scientific and technical expertise. As president, Dr. Chu directs the activities of more than 1,000 scientists and technologists. Together, they conduct and support research requested by federal agencies involved in advancing national security and advising on science and technology issues. Dr. Chu served in the Department of Defense as Under Secretary of Defense for Personnel and Readiness from 2001-2009, and earlier as Assistant Secretary of Defense and Director for Program Analysis and Evaluation from 1981-1993. From 1978-1981 he was the Assistant Director of the Congressional Budget Office for National Security and International Affairs. Dr. Chu served in the U. S. Army from 1968-1970. He was an economist with the RAND Corporation from 1970-1978, director of RAND’s Washington Office from 1994-1998, and vice president for its Army Research Division from 1998-2001. He earned a bachelor of arts in economics and mathematics, and his doctorate in economics, from Yale University. Dr. Chu is a member of the Defense Science Board and a Fellow of the National Academy of Public Administration. He is a recipient of the Department of Defense Medal for Distinguished Public Service with Gold Palm, the Department of Veterans Affairs Meritorious Service Award, the Department of the Army Distinguished Civilian Service Award, the Department of the Navy Distinguished Public Service Award, and the National Academy of Public Administration’s National Public Service Award. |
Keynote | 2019 |
|
Keynote Thursday Keynote Speaker II |
Michael Little Program Manager, Advanced Information Systems Technology Earth Science Technology Office, NASA Headquarters ![]() (bio)
Over the past 45 years, Mike’s primary focus has been on the management of research and development, focusing on making the results more useful in meeting the needs of the user community. Since 1984, he has specialized in communications, data and processing systems, including projects in NASA, the US Air Force, the FAA and the Census Bureau. Before that, he worked on Major System Acquisition Programs, in the Department of Defense including Marine Corps combat vehicles and US Navy submarines. Currently, Mike manages a comprehensive program to provide NASA’s Earth Science research efforts with the information technologies it will need in the 2020-2035 time-frame to characterize, model and understand the Earth. This Program addresses the full range of data lifecycle from generating data using instruments and models, through the management of the data and including the ways in which information technology can help to exploit the data. Of particular interest today are the ways in which NASA can measure and understand transient and transitional phenomena and the impact of climate change. The AIST Program focuses the application of applied math and statistics, artificial intelligence, case-based reasoning, machine learning and automation to improve our ability to use observational data and model output in understanding Earth’s physical processes and natural phenomena. Training and odd skills: Application of cloud computing US Government Computer Security US Navy Nuclear Propulsion operations and maintenance on two submarines |
Keynote |
![]() | 2019 |
Keynote Thursday Keynote Speaker I |
Wendy Martinez Director, Mathematical Statistics Research Center, Bureau of Labor Statistics ASA President-Elect (2020) ![]() (bio)
Wendy Martinez has been serving as the Director of the Mathematical Statistics Research Center at the Bureau of Labor Statistics (BLS) for six years. Prior to this, she served in several research positions throughout the Department of Defense. She held the position of Science and Technology Program Officer at the Office of Naval Research, where she established a research portfolio comprised of academia and industry performers developing data science products for the future Navy and Marine Corps. Her areas of interest include computational statistics, exploratory data analysis, and text data mining. She is the lead author of three books on MATLAB and statistics. Dr. Martinez was elected as a Fellow of the American Statistical Association (ASA) in 2006 and is an elected member of the International Statistical Institute. She was honored by the American Statistical Association when she received the ASA Founders Award at the JSM 2017 conference. Wendy is also proud and grateful to have been elected as the 2020 ASA President. |
Keynote | Materials | 2019 |
Keynote Wednesday Keynote Speaker II |
Laura Freeman Associate Director, ISL Hume Center for National Security and Technology, Virginia Tech ![]() (bio)
Dr. Laura Freeman is an Assistant Director of the Operational Evaluation Division at the Institute for Defense Analyses. In that position, she established and developed an interdisciplinary analytical team of statisticians, psychologists, and engineers to advance scientific approaches to DoD test and evaluation. Her focus areas include test design, statistical data analysis, modeling and simulation validation, human-system interactions, reliability analysis, software testing, and cybersecurity testing. Dr. Freeman currently leads a research task for the Chief Management Officer (CMO) aiming to reform DoD testing. She guides an interdisciplinary team in recommending changes and developing best practices. Reform initiatives include incorporating mission context early in the acquisition lifecycle, integrating all test activities, and improving data management processes. During 2018, Dr. Freeman served as that acting Senior Technical Advisor for Director Operational Test and Evaluation (DOT&E). As the Senior Technical Advisor, Dr. Freeman provided leadership, advice, and counsel to all personnel on technical aspects of testing military systems. She served as a liaison with Service technical advisors, General Officers, and members of the Senior Executive Service on key technical issues. She reviewed test strategies, plans, and reports from all systems on DOT&E oversight. During her tenure at IDA, Dr. Freeman has designed tests and conducted statistical analyses for programs of national importance including weapon systems, missile defense, undersea warfare systems, command and control systems, and most recently the F-35. She prioritizes supporting the analytical community in the DoD workforce. She developed and taught numerous courses on advanced test design and statistical analysis, including two new Defense Acquisition University (DAU) Courses on statistical methods. She is a founding organizer of DATAWorks (Defense and Aerospace Test and Analysis Workshop), a workshop designed to share new methods, provide training, and share best practices between NASA, the DoD, and National Labs. Dr. Freeman is the recipient of the 2017 IDA Goodpaster Award for Excellence in Research and the 2013 International Test and Evaluation Association (ITEA) Junior Achiever Award. She is a member of the American Statistical Association, the American Society for Quality, the International Statistical Engineering Association, and ITEA. She serves on the editorial boards for Quality Engineering, Quality Reliability Engineering International, and the ITEA Journal. Her areas of statistical expertise include designed experiments, reliability analysis, and industrial statistics. Prior to joining IDA in 2010, Dr. Freeman worked at SAIC providing statistical guidance to the Director, Operational Test and Evaluation. She also consulted with NASA on various projects. In 2008, Dr. Freeman established the Laboratory for Interdisciplinary Statistical Analyses at Virginia Tech and Served as its inaugural Director. Dr. Freeman has a B.S. in Aerospace Engineering, a M.S. in Statistics and a Ph.D. in Statistics, all from Virginia Tech. Her Ph.D. research was on design and analysis of experiments for reliability data. |
Keynote |
![]() | 2019 |
Keynote Thursday Lunchtime Keynote Speaker |
T. Charles Clancy Bradley Professor of Electrical and Computer Engineering Virginia Tech ![]() (bio)
Charles Clancy is the Bradley Professor of Electrical and Computer Engineering at Virginia Tech where he serves as the Executive Director of the Hume Center for National Security and Technology. Clancy leads a range of strategic programs at Virginia Tech related to security, including the Commonwealth Cyber Initiative. Prior to joining VT in 2010, Clancy was an engineering leader in the National Security Agency, leading research programs in digital communications and signal processing. He received his PhD from the University of Maryland, MS from University of Illinois, and BS from the Rose-Hulman Institute of Technology. He is co-author to over 200 peer-reviewed academic publications, six books, over twenty patents, and co-founder to five venture-backed startup companies. |
Keynote |
![]() | 2019 |
Keynote Wednesday Keynote Speaker III |
Timothy Dare Deputy Director, Developmental Test, Evaluation, and Prototyping SES OUSD(R&E) ![]() (bio)
Mr. Timothy S. Dare is the Deputy Director for Developmental Test, Evaluation and Prototyping (DD(DTEP)). As the DD(DTEP), he serves as the principal advisor on developmental test and evaluation (DT&E) to the Secretary of Defense, Under Secretary of Defense for Research and Engineering, and Director of Defense Research and Engineering for Advanced Capabilities. Mr. Dare is responsible for DT&E policy and guidance in support of the acquisition of major Department of Defense (DoD) systems, and providing advocacy, oversight, and guidance to the DT&E acquisition workforce. He informs policy and advances leading edge technologies through the development of advanced technology concepts, and developmental and operational prototypes. By working closely with interagency partners, academia, industry and governmental labs, he identifies, develops and demonstrates multi-domain technologies and concepts that address high-priority DoD, multi-Service, and Combatant Command warfighting needs. Prior to his appointment in December 2018, Mr. Dare was a Senior Program Manager for program management and capture at Lockheed Martin (LM) Space. In this role he was responsible for the capture and execution phases of multiple Intercontinental Ballistic Missile programs for Minuteman III, including a new airborne Nuclear Command and Control (NC2) development program. His major responsibilities included establishing program working environments at multiple locations, policies, processes, staffing, budget and technical baselines. Mr. Dare has extensive T&E and prototyping experience. As the Engineering Program Manager for the $1.8B Integrated Space C2 programs for NORAD/NORTHCOM systems at Cheyenne Mountain, Mr. Dare was the Integration and Test lead focusing on planning, executing, and evaluating the integration and test phases (developmental and operational T&E) for Missile Warning and Space Situational Awareness (SSA) systems. Mr. Dare has also been the Engineering Lead/Integration and Test lead on other systems such as the Hubble Space Telescope; international border control systems; artificial intelligence (AI) development systems (knowledge-based reasoning); Service-based networking systems for the UK Ministry of Defence; Army C2 systems; Space Fence C2; and foreign intelligence, surveillance, and reconnaissance systems. As part of the Department’s strategic defense portfolio, Mr. Dare led the development of advanced prototypes in SSA C2 (Space Fence), Information Assurance (Single Sign-on), AI systems, and was the sponsoring program manager for NC2 capability development. Mr. Dare is a graduate of Purdue University and is a member of both the Association for Computing Machinery and Program Management Institute. He has been recognized by the U.S. Air Force for his contributions supporting NORAD/NORTHCOM’s strategic defense missions, and the National Aeronautics and Space Administration for his contributions to the original Hubble Space Telescope program. Mr. Dare holds a U.S. Patent for Single Sign-on architectures. |
Keynote |
![]() | 2019 |
Keynote Wednesday Lunchtime Keynote Speaker |
Jared Freeman Chief Scientist of Aptima and Chair of the Human Systems Division National Defense Industry Association ![]() (bio)
Jared Freeman, Ph.D., is Chief Scientist of Aptima and Chair of the Human Systems Division of the National Defense Industry Association. His research and publications address measurement, assessment, and enhancement of human learning, cognition, and performance in technologically complex military environments. |
Keynote |
![]() | 2019 |
Keynote Welcoming & Opening Keynote-Tuesday AM |
Mike Gilmore Director DOT&E ![]() (bio)
Link to Bio unavail |
Keynote | Materials | 2016 |
Keynote Lunch with Keynote Leadership Perspective |
Jon Hollday NASA ![]() (bio)
Link to Bio unavail |
Keynote | 2016 |
|
Keynote STAT Engineering Keynote-Wednesday AM |
Christine Anderson Cook Statistics Los Alamos National Lab ![]() (bio)
s in the areas of complex system reliability, non-proliferation, malware detection and statistical process control. Before joining LANL, she was a faculty member in the Department of Statistics at Virginia Tech for 8 years. Her research areas include response surface methodology, design of experiments, reliability, multiple criterion optimization and graphical methods. She has authored more than 130 articles in statistics and quality peerreviewed journals, and has been a long time contributor to the Quality Progress Statistics Roundtable column. In 2012, she edited a special issue in Quality Engineering on Statistical Engineering with Lu Lu. She is an elected fellow of the American Statistical Association and the American Society for Quality. In 2012 she was honored with the ASQ Statistics Division William G. Hunter Award. In 2011 she received the 26th Annual Governor’s Award for Outsta |
Keynote | 2016 |
|
Keynote Lunch With Leadership Perspective-Wednesday PM |
David Brown Deputy Assistant Secretary of Defense, Developmental Test & Evaluation ![]() (bio)
Link to Bio unavail |
Keynote | 2016 |
|
Keynote Test and Evaluation Matter for Warfighter and We Need your Help to Make it Better |
Katherine Warner Science Advisor within Secretary of Defense DOT&E ![]() (bio)
Dr. Catherine Warner serves as the Science Advisor for the Director, Operational Test and Evaluation within the Office of the Secretary of Defense. Dr. Warner has been involved with operational test and evaluation since 1991, when she became a research staff member at the Institute for Defense Analyses (IDA). In that capacity, Dr. Warner performed and directed analysis of operational tests for Army, Navy, and Air Force systems in support of DOT&E. From 2005 – 2010, Dr. Warner was an Assistant Director at IDA and also served as the lead for the Air Warfare group. Her analysis portfolio included major aircraft systems such as the F-22, F/A-18E/F, V-22, and H-1. Prior to that, Dr. Warner was the lead analyst for Unmanned Aerial Vehicle (UAV) systems including Predator, Shadow, Hunter, and Global Hawk. In 2013 at the request of the Defense Information Systems Agency (DISA), Dr. Warner deployed to Kabul, Afghanistan for 16 months in support of NATO’s International Security Assistance Force (ISAF) and the US Operation Enduring Freedom (OEF). She lead a team of Information Technology specialists in advising Afghanistan’s Ministry of Communications and Information Technology on enhancing national communication capabilities for the security and economic growth of the country. The primary focus of this team included supporting the completion of Afghanistan’s National Fiber Optic Ring, Spectrum Management, and Cyber Security.Dr. Warner previously worked at the Lawrence Livermore National Laboratory. She grew up in Albuquerque, New Mexico, attended the University of New Mexico and San Jose State University where she earned both B.S. and M.S. degrees in Chemistry. She also earned both M.A. and Ph.D. degrees in Chemistry from Princeton University. |
Keynote | Materials | 2017 |
Keynote Retooling Design and Development |
Chris Singer NASA Deputy Chief Engineer NASA ![]() (bio)
Christopher (Chris) E. Singer is the NASA Deputy, Chief Engineer responsible for integrating engineering across the Agencies 10 field centers. Prior to this appointment in April 2016, he served as the Engineering Director at NASA’s Marshall Space Flight Center in Huntsville, Alabama. Appointed in 2011, Mr. Singer led an organization of 1,400 civil service and 1,200 support contractor employees responsible for the design, testing, evaluation, and operation of hardware and software associated with space transportation, spacecraft systems, science instruments and payloads under development at the Marshall Center. The Engineering Directorate also manages NASA’s Payload Operations Center at Marshall, which is the command post for scientific research activities on-board the International Space Station. Mr. Singer began his NASA career in 1983 as a rocket engine specialist. In 1992, he served a one-year assignment at NASA Headquarters in Washington, DC, as senior manager for the space shuttle main engine and external tank in the Space Shuttle Support Office. In 1994, Mr. Singer supervised the development and implementation of safety improvements and upgrades to shuttle propulsion components. In 2000, he was appointed chief engineer in the Space Transportation Directorate then was selected as deputy director of Marshall’s Engineering Directorate from 2004 to 2011. Mr. Singer is an AIAA Associate Fellow. In 2006, he received the Presidential Rank Award for Meritorious Executives — the highest honor for career federal employees. He was awarded the NASA Outstanding Leadership Medal in 2001 and 2008 for his leadership. In 1989, he received the prestigious Silver Snoopy Award from the Astronaut Corps for his contributions to the success of human spaceflight missions. A native of Nashville, Tennessee, Mr. Singer earned a bachelor’s degree in mechanical engineering in 1983 from Christian Brothers University in Memphis, Tennessee. Chris enjoys woodworking, fishing and Hang gliding. Chris is married to the former Jody Adams of Hartselle, Alabama. They have three children and live in Huntsville, Alabama. |
Keynote | Materials | 2017 |
Keynote Reflections on Statistical Engineering and it’s Application |
Geoff Vining Professor Virginia Tech ![]() (bio)
Geoff Vining is a Professor of Statistics at Virginia Tech. From 1999 – 2006, he also was the department head. He currently is the ASQ Treasurer for 2016 and Past-Chair of the ASQ Technical Communities Council. He is a Fellow of the ASQ, a Fellow of the American Statistical Association (ASA), and an Elected Member of the International Statistical Institute. Dr. Vining served as Editor of the Journal of Quality Technology from 1998 – 2000 and as Editor-in-Chief of Quality Engineering from 2008-2009. He also has served as Chair of the ASQ Publications Management Board, as Chair of the ASQ Statistics Division, and as Chair of the ASA Quality and Productivity Section. Dr. Vining won the 2010 Shewhart Medal, the ASQ career award given annually to the person not previously so honored who has demonstrated the most outstanding technical leadership in the field of modern quality control, especially through the development to its theory, principles, and techniques. He also received the 2015 Box Medal from the European Network for Business and Industrial Statistics (ENBIS). This medal recognizes each year an extraordinary statistician who has remarkably contributed with his/her work to the development and the application of statistical methods in European business and industry. In 2013, he received an Engineering Excellence Award from the NASA Engineering and Safety Center. He received the 2011 William G. Hunter Award from the ASQ Statistics Division for excellence in statistics as a communicator, a consultant, an educator, an innovator, an integrator of statistics with other disciplines and an implementer who obtains meaningful results. He won the 1990 ASQ Brumbaugh Award for the paper published in an ASQ journal that made the greatest contribution to the development of industrial applications of quality control and the 2005 Lloyd Nelson Award from the Statistics Division for the paper published in the Journal of Quality Technology that had the greatest immediate impact to practitioners. Dr. Vining is the author of three textbooks. He is an internationally recognized expert in the use of experimental design for quality, productivity, and reliability improvement and in the application of statistical process control. He has extensive consulting experience, most recently with NASA and the U.S. Department of Defense. |
Keynote | Materials | 2017 |
Keynote Developmental Test and Evaluation |
Brian Hall Principal Deputy Director DoD ![]() (bio)
Dr. Hall was appointed to the Senior Executive Service in November 2014 as the Principal Deputy Director for Developmental Test and Evaluation, and is currently serving as the Principle Deputy Director of the Test Resource Management Center (TRMC). In this position, he oversees matters concerning the Nation’s critical test range infrastructure, science and technology efforts, development of the biennial Strategic Plan for DoD Test and Evaluation (T&E) resources, as well as certification of the Services’ T&E budgets. Prior to this position, Dr. Hall was the Technical Advisor for Operational Test and Evaluation (OT&E) of all Land and Expeditionary Warfare systems in OSD-DOT&E. In this position, advised the highest authorities in DoD for OT&E, observed operational testing, and coauthored Operational Assessments and Beyond Low-Rate Initial Production reports submitted to the 4 Congressional Defense Committees. Prior to serving on the OSD staff, Dr. Hall was the Division Chief for Aviation, Missiles, and C4ISR Systems in the Army Test and Evaluation Command (ATEC), where he was responsible for supervising the development of test plans and evaluations that directly supported milestone decision reviews and materiel fielding/production decisions of more than 300 Army programs. While with the Army, Dr. Hall was one of the leading reliability experts that helped establish the Center for Reliability Growth at Aberdeen Proving Ground, as well as develop and administer the ATEC/AMSAA 3-day reliability course to improve defense acquisition learning. Over his career, Dr. Hall has led studies, developed methodologies, presented research, published papers, crafted policy, and authored policy implementation guides. He has also developed staff, advised numerous defense programs, and served as an executive member of, or invited contributor to: tri-service DoD Blue Ribbon Panels; National Academy of Science studies; and DoD working-groups to improve system reliability. Dr. Hall is a Senior Service College graduate and has earned advanced degrees in Applied Mathematics, Reliability Engineering, and Strategic Studies. He is a domain expert in Reliability Engineering, as well as in Reliability Growth Management and Methodology. He has developed statistical methods and reliability growth models that have been: published in international journals, incorporated into Military Handbooks, adopted by Operational Test Agency policy, and utilized to shape growth plans and assess reliability maturity of numerous DoD systems. |
Keynote | Materials | 2017 |
Keynote
|
Dave Duma Acting Director, Operationak Test and Evaluation, Office of the Secretary of Defense DOT&E ![]() (bio)
Mr. Duma is the Acting Director, Operational Test and Evaluation as of January 20, 2017. Mr. Duma was appointed as the Principal Deputy Director, Operational Test and Evaluation in January 2002. In this capacity he is responsible for all functional areas assigned to the office. He participates in the formulation, development, advocacy, and oversight of policies of the Secretary of Defense and in the development and implementation of test and test resource programs. He oversees the planning, conduct, analysis, evaluation, and reporting of operational and live fire testing. He serves as the Appropriation Director and Comptroller for the Operational Test and Evaluation, Defense Appropriation and coordinates all Planning, Programming, and Budgeting Execution matters. He previously served as Acting Director, Operational Test and Evaluation from February 2005 to July 2007 and again from May 2009 to September 2009. Mr. Duma also served as the Acting Deputy Director, Operational Test and Evaluation from January 1992 to June 1994. In this capacity he was responsible for oversight of the planning, conduct, analysis, and reporting of operational test and evaluation for all major conventional weapons systems in the Department of Defense. He supervised the development of evaluation plans and test program strategies, observed the conduct of operational test events, evaluated operational field tests of all armed services and submitted final reports for Congress. Mr. Duma returned to government service from the commercial sector. In private industry he worked a variety of projects involving test and evaluation; requirements generation; command, control, communications, intelligence, surveillance and reconnaissance; modeling and simulation; and software development. Mr. Duma has 30 years of naval experience during which he was designated as a Joint Service Officer. He served as the Director, Test and Evaluation Warfare Systems for the Chief of Naval Operations, the Deputy Commander, Submarine Squadron TEN, and he commanded the nuclear powered submarine USS SCAMP (SSN 588). Mr. Duma holds Masters of Science degrees in National Security and Strategic Studies and in Management. He holds a Bachelor of Science degree in Nuclear Engineering. He received the U.S. Presidential Executive Rank Award on two occasions; in 2008, the Meritorious Executive Award and in 2015, the Distinguished Executive Rank Award. He is a member of the International Test and Evaluation Association. |
Keynote | 2017 |
|
Keynote Opening Keynote |
David Chu President IDA ![]() (bio)
David Chu serves as President of the Institute for Defense Analyses. IDA is a non-profit corporation operating in the public interest. Its three federally funded research and development centers provide objective analyses of national security issues and related national challenges, particularly those requiring extraordinary scientific and technical expertise. As president, Dr. Chu directs the activities of more than 1,000 scientists and technologists. Together, they conduct and support research requested by federal agencies involved in advancing national security and advising on science and technology issues. Dr. Chu served in the Department of Defense as Under Secretary of Defense for Personnel and Readiness from 2001-2009, and earlier as Assistant Secretary of Defense and Director for Program Analysis and Evaluation from 1981-1993. From 1978-1981 he was the Assistant Director of the Congressional Budget Office for National Security and International Affairs. Dr. Chu served in the U. S. Army from 1968-1970. He was an economist with the RAND Corporation from 1970-1978, director of RAND’s Washington Office from 1994-1998, and vice president for its Army Research Division from 1998-2001. He earned a bachelor of arts in economics and mathematics, and his doctorate in economics, from Yale University. Dr. Chu is a member of the Defense Science Board and a Fellow of the National Academy of Public Administration. He is a recipient of the Department of Defense Medal for Distinguished Public Service with Gold Palm, the Department of Veterans Affairs Meritorious Service Award, the Department of the Army Distinguished Civilian Service Award, the Department of the Navy Distinguished Public Service Award, and the National Academy of Public Administration’s National Public Service Award. |
Keynote |
![]() | 2018 |
Keynote Closing Remarks |
Robert Behler Director DOT&E ![]() (bio)
Robert F. Behler was sworn in as Director of Operational Test and Evaluation on December 11, 2017. A Presidential appointee confirmed by the United States Senate, he serves as the senior advisor to the Secretary of Defense on operational and live fire test and evaluation of Department of Defense weapon systems. Prior to his appointment, he was the Chief Operating Officer and Deputy Director of the Carnegie Mellon University Software Engineering Institute (SEI), a Federally Funded Research and Development Center. SEI is a global leader in advancing software development and cybersecurity to solve the nation’s toughest problems through focused research, development, and transition to the broader software engineering community. Before joining the SEI, Mr. Behler was the President and CEO of SRC, Inc. (formerly the Syracuse Research Corporation). SRC is a not-for-profit research and development corporation with a forprofit manufacturing subsidiary that focuses on radar, electronic warfare and cybersecurity technologies. Prior to working at SRC, Mr. Behler was the General Manager and Senior Vice President of the MITRE Corp where he provided leadership to more than 2,500 technical staff in 65 worldwide locations. He joined MITRE from the Johns Hopkins University Applied Physics Laboratory where he was a General Manager for more than 350 scientists and engineers as they made significant contributions to critical Department of Defense (DOD) precision engagement challenges. General Behler served 31 years in the United States Air Force, retiring as a Major General in 2003. During his military career, he was the Principal Adviser for Command and Control, Intelligence, Surveillance and Reconnaissance (C21SR) to the Secretary and Chief of Staff of the U.S. Air Force (USAF). International assignments as a general officer included the Deputy Commander for NATO’s Joint Headquarters North in Stavanger, Norway. He was the Director of the Senate Liaison Office for the USAF during the 104th congress. Mr. Behler also served as the assistant for strategic systems to the Director of Operational Test and Evaluation. As an experimental test pilot, he flew more than 65 aircraft types. Operationally he flew worldwide reconnaissance missions in the fastest aircraft in the world, the SR-71 Blackbird. Mr. Behler is a Fellow of the Society of Experimental Test Pilots and an Associate Fellow of the American Institute of Aeronautics and Astronautics. He is a graduate of the University of Oklahoma where he received a B.S. and M.S. in aerospace engineering, has a MBA from Marymount University and was a National Security Fellow at the JFK School of Government at Harvard University. Mr. Behler has recently been on several National Research Council studies for the National Academy of Sciences including: “Critical Code,” “Software Producibility, Achieving Effective Acquisition of Information Technology in the Department of Defense” and “Development Planning: A Strategic Approach to Future Air Force Capabilities.” |
Keynote | 2018 |
|
Keynote Testing and Analytical Challenges on the Path to Hypersonic Flight |
Mark Lewis Director IDA-STPI ![]() (bio)
Dr. Mark J. Lewis is the Director of IDA’s Science and Technology Policy Institute, a federally funded research and development center. He leads an organization that provides analysis of national and international science and technology issues to the Office of Science and Technology Policy in the White House, as well as other Federal agencies including the National Institutes of Health, the National Science Foundation, NASA, the Department of Energy, Homeland Security, and the Federal Aviation Administration, among others.Prior to taking charge of STPI, Dr. Lewis served as the Willis Young, Jr. Professor and Chair of the Department of Aerospace Engineering at the University of Maryland. A faculty member at Maryland for 24 years, Dr. Lewis taught and conducted basic and applied research. From 2004 to 2008, Dr. Lewis was the Chief Scientist of the U.S. Air Force. From 2010 to 2011, he was President of the American Institute of Aeronautics and Astronautics (AIAA).Dr. Lewis attended the Massachusetts Institute of Technology, where he received a Bachelor of Science degree in aeronautics and astronautics, Bachelor of Science degree in earth and planetary science (1984), Master of Science (1985), and Doctor of Science (1988) in aeronautics and astronautics.Dr. Lewis is the author of more than 300 technical publications and has been an adviser to more than 70 graduate students. Dr. Lewis has also served on various advisory boards for NASA, the Air Force, and DoD, including two terms on the Air Force Scientific Advisory Board, the NASA Advisory Council, and the Aeronautics and Space Engineering Board of the National Academies.Dr. Lewis’s awards include the Meritorious Civilian Service Award and Exceptional Civilian Service Award; he was also recognized as the 1994 AIAA National Capital Young Scientist/Engineer of the Year, IECEC/ AIAA Lifetime Achievement Award, and is an Aviation Week and Space Technology Laureate (2007). |
Keynote |
![]() | 2018 |
Keynote Journey to a Data Centric Approcach for National Security |
Marcey Hoover Quality Assurance Director Sandia National Labortories ![]() (bio)
As Quality Assurance Director, Dr. Marcey Hoover is responsible for designing and sustaining the Laboratories’ quality assurance system and the associated technical capabilities needed for flawless execution of safe, secure, and efficient work to deliver exceptional products and services to its customers.Marcey previously served as the Senior Manager responsible for developing the science and engineering underpinning efforts to predict and influence the behavior of complex, highly interacting systems critical to our nation’s security posture. In her role as Senior Manager and Chief of Operations for Sandia’s Energy and Climate program, Marcey was responsible for strategic planning, financial management, business development, and communications. In prior positions, she managed organizations responsible for (1) quality engineering on new product development programs, (2) research and development of advanced computational techniques in the engineering sciences, and (3) development and execution of nuclear weapon testing and evaluation programs. Marcey has also led several executive office functions, including corporate- level strategic planning.Active in both the American Statistical Association and the American Society for Quality (ASQ), Marcey served two terms as the elected ASQ Statistics Division Treasurer. She was recognized as the Outstanding Alumni of the Purdue University Statistics Department in 2009 and nominated in 2011 for the YWCA Middle Rio Grande Women on the Move award. She currently serves on both the Purdue Strategic Research Advisory Council and the Statistics Alumni Advisory Board, and as a mentor for Big Brothers Big Sisters of New Mexico.Marcey received her bachelor of science degree in mathematics from Michigan State University, and her master of science and doctor of philosophy degrees in mathematical statistics from Purdue University. |
Keynote |
![]() | 2018 |
Keynote NASA AERONAUTICS |
Bob Pearce Deputy Associate Administrator Strategy, Aeronautics Research Mission DirectorateNASA ![]() (bio)
Mr. Pearce is responsible for leading aeronautics research mission strategic planning to guide the conduct of the agency’s aeronautics research and technology programs, as well as leading ARMD portfolio planning and assessments, mission directorate budget development and approval processes, and review and evaluation of all of NASA’s aeronautics research mission programs for strategic progress and relevance. Pearce is also currently acting director for ARMD’s Airspace Operations and Safety Program, and responsible for the overall planning, management and evaluation of foundational air traffic management and operational safety research. Previously he was director for strategy, architecture and analysis for ARMD, responsible for establishing a strategic systems analysis capability focused on understanding the system-level impacts of NASA’s programs, the potential for integrated solutions, and the development of high-leverage options for new investment and partnership. From 2003 until July 2010, Pearce was the deputy director of the FAA-led Next Generation Air Transportation System (NextGen) Joint Planning and Development Office (JPDO). The JPDO was an interagency office tasked with developing and facilitating the implementation of a national plan to transform the air transportation system to meet the long-term transportation needs of the nation. Prior to the JPDO, Pearce held various strategic and program management positions within NASA. In the mid-1990s he led the development of key national policy documents including the National Science and Technology Council’s “Goals for a National Partnership in Aeronautics Research and Technology” and the “Transportation Science and Technology Strategy.” These two documents provided a substantial basis for NASA’s expanded investment in aviation safety and airspace systems. He began his career as a design engineer at the Grumman Corporation, working on such projects as the Navy’s F-14 Tomcat fighter and DARPA’s X-29 Forward Swept Wing Demonstrator. Pearce also has experience from the Department of Transportation’s Volpe National Transportation Systems Center where he made contributions in the area of advanced concepts for intercity transportation systems. Pearce has received NASA’s Exceptional Service Medal for sustained excellence in planning and advocating innovative aeronautics programs in conjunction with the White House and other federal agencies. He received NASA’s Exceptional Achievement Medal for outstanding leadership of the JPDO in support of the transformation of the nation’s air transportation system. Pearce has also received NASA’s Cooperative External Achievement Award and several Exceptional Performance and Group Achievement Awards.He earned a bachelor’s of science degree in mechanical and aerospace engineering from Syracuse University, and a master’s of science degree in technology and policy from the Massachusetts Institute of Technology. |
Keynote | Materials | 2018 |
Session Title | Speaker | Type | Materials | Year |
---|---|---|---|---|
Panel Finding the Human in the Loop: Considerations for AI in Decision Making |
Joe Lyons Lead for the Collaborative Interfaces and Teaming Core Research Area 711 Human Performance Wing at Wright-Patterson AFB ![]() |
Panel |
![]() Recording | 2021 |
Panel Finding the Human in the Loop: Panelist |
Chad Bieber Director, Test and Evaluation. Senior Research Engineer. Johns Hopkins University Applied Physics Laboratory ![]() |
Panel |
Recording | 2021 |
Panel Finding the Human in the Loop: Panelist |
Poornima Madhavan Principal Scientist and Capability Lead for Social and Behavioral Sciences MITRE ![]() |
Panel |
Recording | 2021 |
Keynote Wednesday Keynote Speaker I |
Peter Parker Team Lead, Advanced Measurement Systems NASA ![]() |
Keynote |
![]() | 2019 |
Keynote Tuesday Keynote |
David Chu President Institute for Defense Analyses ![]() |
Keynote | 2019 |
|
Keynote Thursday Keynote Speaker II |
Michael Little Program Manager, Advanced Information Systems Technology Earth Science Technology Office, NASA Headquarters ![]() |
Keynote |
![]() | 2019 |
Keynote Thursday Keynote Speaker I |
Wendy Martinez Director, Mathematical Statistics Research Center, Bureau of Labor Statistics ASA President-Elect (2020) ![]() |
Keynote | Materials | 2019 |
Keynote Wednesday Keynote Speaker II |
Laura Freeman Associate Director, ISL Hume Center for National Security and Technology, Virginia Tech ![]() |
Keynote |
![]() | 2019 |
Keynote Thursday Lunchtime Keynote Speaker |
T. Charles Clancy Bradley Professor of Electrical and Computer Engineering Virginia Tech ![]() |
Keynote |
![]() | 2019 |
Keynote Wednesday Keynote Speaker III |
Timothy Dare Deputy Director, Developmental Test, Evaluation, and Prototyping SES OUSD(R&E) ![]() |
Keynote |
![]() | 2019 |
Keynote Wednesday Lunchtime Keynote Speaker |
Jared Freeman Chief Scientist of Aptima and Chair of the Human Systems Division National Defense Industry Association ![]() |
Keynote |
![]() | 2019 |
Keynote Welcoming & Opening Keynote-Tuesday AM |
Mike Gilmore Director DOT&E ![]() |
Keynote | Materials | 2016 |
Keynote Lunch with Keynote Leadership Perspective |
Jon Hollday NASA ![]() |
Keynote | 2016 |
|
Keynote STAT Engineering Keynote-Wednesday AM |
Christine Anderson Cook Statistics Los Alamos National Lab ![]() |
Keynote | 2016 |
|
Keynote Lunch With Leadership Perspective-Wednesday PM |
David Brown Deputy Assistant Secretary of Defense, Developmental Test & Evaluation ![]() |
Keynote | 2016 |
|
Keynote Test and Evaluation Matter for Warfighter and We Need your Help to Make it Better |
Katherine Warner Science Advisor within Secretary of Defense DOT&E ![]() |
Keynote | Materials | 2017 |
Keynote Retooling Design and Development |
Chris Singer NASA Deputy Chief Engineer NASA ![]() |
Keynote | Materials | 2017 |
Keynote Reflections on Statistical Engineering and it’s Application |
Geoff Vining Professor Virginia Tech ![]() |
Keynote | Materials | 2017 |
Keynote Developmental Test and Evaluation |
Brian Hall Principal Deputy Director DoD ![]() |
Keynote | Materials | 2017 |
Keynote
|
Dave Duma Acting Director, Operationak Test and Evaluation, Office of the Secretary of Defense DOT&E ![]() |
Keynote | 2017 |
|
Keynote Opening Keynote |
David Chu President IDA ![]() |
Keynote |
![]() | 2018 |
Keynote Closing Remarks |
Robert Behler Director DOT&E ![]() |
Keynote | 2018 |
|
Keynote Testing and Analytical Challenges on the Path to Hypersonic Flight |
Mark Lewis Director IDA-STPI ![]() |
Keynote |
![]() | 2018 |
Keynote Journey to a Data Centric Approcach for National Security |
Marcey Hoover Quality Assurance Director Sandia National Labortories ![]() |
Keynote |
![]() | 2018 |
Keynote NASA AERONAUTICS |
Bob Pearce Deputy Associate Administrator Strategy, Aeronautics Research Mission DirectorateNASA ![]() |
Keynote | Materials | 2018 |