Klein, E. 2023. “The surprising thing A.I. engineers will tell you if you let them.” The New York Times. 16 April 2023. https://www.nytimes.com/2023/04/16/opinion/this-is-too- important-to-leave-to-microsoft-google-and-facebook.html. Lindgren, E. 2023. “Validated and deployable AI/ML for NDT data diagnostics.” Materials Evaluation 81 (7): 35–42. Metz, C., and K. Collins. 2023. “10 ways GPT-4 is impressive but still flawed.” The New York Times. 14 March 2023. https:// www.nytimes.com/2023/03/14/technology/openai-new-gpt4. html. Molla, R. 2023. “AI is flooding the workplace, and workers love it.” Vox. 10 April 2023. https://www.vox.com/tech- nology/23673018/generative-ai-chatgpt-bing-bard-work-jobs. Ross, J. 2023. “AI and the future of publishing.” Materials Evaluation 81 (2): 7. Singh, R., and V. Garg. 2023. “Nurturing generative AI: balancing innovation and responsibility.” Materials Evaluation 81 (7): 17–20. Singh, R., V. Garg, and GPT-3. 2021. “Human factors in NDE 4.0 development decisions.” Journal of Nondestructive Evalua- tion 40. https://doi.org/10.1007/s10921-021-00808-3. Timothy, M. 2023. “9 practical uses of ChatGPT in program- ming,” MUO. 20 March 2023. https://www.makeuseof.com/ chatgpt-programming-practical-uses/. Vrana, J., R. Singh, ChatGPT. 2023. “This is ChatGPT how may I help you?” Materials Evaluation, 81 (2) 17–18. Wolfram, S. 2023. “What is ChatGPT doing…and why does it work?” Stephen Wolfram Writings. 14 February 2023. https:// writings.stephenwolfram.com/2023/02/what-is-chatgpt -doing-and-why-does-it-work/. Yang, H. 2023. “How I use ChatGPT responsibly in my teaching.” Nature. 12 April 2023. https://doi.org/10.1038/ d41586-023-01026-9. FEATURE |AI/ML 34 M A T E R I A L S E V A L U A T I O N • J U L Y 2 0 2 3 2307 ME July dup.indd 34 6/19/23 3:41 PM

AI/ML |FEATURE VALIDATED AND DEPLOYABLE AI/ML FOR NDT DATA DIAGNOSTICS BY ERIC LINDGREN While artificial intelligence/machine learning (AI/ML) methods have shown promise for the analysis of image and signal data, applications using nondestructive testing (NDT) for managing the safety of systems must meet a high level of quantified capability. Engineering decisions require technique validation with statistical bounds on performance to enable integration into critical analyses, such as life management and risk analysis. The Air Force Research Laboratory (AFRL) has pursued several projects to apply a hybrid approach that integrates AI/ML methods with heuristic and model-based algorithms to assist inspectors in accomplishing complex NDT evaluations. Three such examples are described in this article, including a method that was validated through a probability of detection (POD) study and deployed by the Department of the Air Force (DAF) in 2004 (Lindgren et al. 2005). Key lessons learned include the importance of considering the wide variability present in NDT applications upfront and maintaining a critical role for human inspectors to ensure NDT data quality and address outlier indications. Introduction There is a growing increase in interest and attention in AI/ML, which are statistical methods for data analysis. The promise of AI/ML is to use statistical methods to self-extract attributes in the data, such as relationships and/or trends in data that are not as quickly and reliably made through typical human observation. The DAF has embraced the use of these tools for applications where it can accelerate decision-making in representative campaigns, as shown in Figure 1. The objective defined for one of these efforts is summarized as: “The Air Force aims to harness and wield the most optimal forms of artificial intelligence to accomplish all mission-sets of the service with greater speed and accuracy” (USAF n.d.). With the potential to secure more NDT data through the transformation to fully digital instru- ments connected as envisioned by the Internet of Things (IoT) and NDE 4.0, there is an increased interest to use AI/ML methods as the diagnostic tool to determine if a flaw is present in NDT data. Justification for the use of AI/ML includes improved accuracy, improved reliability, and faster disposi- tion time by decreasing or eliminating dependence on human interpretation and analysis of NDT data. The initial focus for the use of AI/ML addresses the detection of flaw indications, although there is exploration in the use of AI/ML to provide addi- tional information on characterizing the size and location of discontinuities. When considering the applicability of AI/ML for flaw detection, it is important to recall that these technical approaches are based on statistical methods, namely regression or classification of data. The concept includes the use of multiple statistical methods in parallel combined with multiple layers of analysis to extract statistical trends in the data to enable decisions that are not readily detectable through more classical methods. These multidimen- sional data analysis methods frequently are called neural networks. These approaches can either be trained using data with known ground truths called supervised AL/ML, or be allowed to form the sta- tistical relationships without training data, called unsupervised AI/ML. As these methods rely on Figure 1. The Department of the Air Force artificial intelligence/machine learning campaign illustration. J U L Y 2 0 2 3 • M A T E R I A L S E V A L U A T I O N 35 2307 ME July dup.indd 35 6/19/23 3:41 PM US AIR FORCE GRAPHIC BY TRAVIS BURCHAM