TIPS FOR EFFECTIVE MACHINE LEARNING IN NDT/E BY JOEL B. HARLEY, SUHAIB ZAFAR, AND CHARLIE TRAN The proliferation of machine learning (ML) advances will have long- lasting effects on the nondestructive testing/evaluation (NDT/E) community. As these advances impact the field and as new datasets are created to support these methods, it is important for researchers and practitioners to understand the associated challenges. This article provides basic definitions from the ML literature and tips for nondestructive researchers and practitioners to choose an ML architecture and to understand its relationships with the associated data. By the conclusion of this article, the reader will be able to identify the type of ML architecture needed for a given problem, be aware of how characteristics of the data affect the architecture’s training, and understand how to evaluate the ML performance based on properties of the dataset. Introduction Advances in ML have consistently gen- erated headlines in the past few years. These developments can be attributed to sophisticated algorithms, faster hardware, and reduced costs for data storage. The natural consequence of such advancements is the deluge of datasets, often known as the age of big data. ML algorithms, especially deep learning, capitalize on these foundations, finding applications in speech recogni- tion and object detection while opening up new possibilities through innovations such as ChatGPT (OpenAI 2023). These applications vary considerably from one another, yet the main task in each case is to recognize patterns in datasets. Pattern recognition is arguably the primary driving force behind new sci- entific and engineering discoveries. For instance, Kepler utilized the observa- tions of Tycho Brahe in astronomy to derive the laws governing planetary motion, which formed the basis for classical mechanics (Bishop 2006). However, data was not a driving force behind scientific inquiry until recently (Brunton et al. 2020), and these trends have also impacted NDT/E (Taheri et al. 2022), with recent advances such as crack detection in concrete using neural networks (Saleem and Gutierrez 2021) or identifying damage modes in compos- ite structures via clustering algorithms (Xu et al. 2020). Neural networks are one of the most widely used algorithms today and can be understood as a class of mathematical models inspired by the structure of the human brain. However, utilizing neural networks, or ML in general, for tasks such as defect detection or aiding data interpretation is a familiar trend in NDT/E. Martín et al. (2007) published a study in 2007 to interpret ultrasonic oscillograms obtained via the pulse-echo method with the aid of neural networks. Even earlier, in the 1990s, Mann et al. (1992) presented the use of neural networks to classify ultrasonic signals obtained from microfiber cracking in a specimen built using a metal matrix composite. These examples demonstrate that the NDT/E community has long recognized the need to augment human judgment with pattern recognition algorithms. Despite these advances, limita- tions of ML in NDT/E have mitigated its impact on the field when compared with other disciplines. A widely acknowl- edged problem is the limited amounts of data available, which is the driving force behind the success of ML in many applications. Even if the lack of training data is not an issue with data-intensive applications, such as acoustic emission testing (Sikorska and Mba 2008), acquir- ing data with a high signal-to-noise ratio (SNR) is a significant hurdle. Finally, an adequate level of understanding and experience in ML techniques is required to ensure the accurate performance of algorithms, which currently needs improvement (Vejdannik et al. 2019). In this article, we address important challenges in applying ML to NDT/E by providing guidelines for practitioners and researchers on building high-quality datasets and using appropriate algo- rithms to ensure high performance from trained ML models. The desired outcome of this effort is to encourage progress in realizing the full poten- tial of ML in NDT/E, leading to more accurate and efficient testing methods in the future. Note that the focus of this article is on how to assess datasets and results. Detailed descriptions of the ML algorithms can be found in other papers (Taheri and Zafar 2023). Forms of Machine Learning ML can be divided into various learning paradigms, each with its characteristics and uses. Below are descriptions for two of these paradigms: supervised learning and unsupervised learning. Examples of supervised learning and unsupervised learning are illustrated in Figure 1. Supervised learning: An ML paradigm that trains the parameters (often numerical weights) of a model from input data (features) and known output data (labels). Supervised learning is the most popular ML paradigm due to the ease at which model training can be directly translated to the target task. The key element of supervised learning is the availability of labeled data. Yet in AI/ML |NDT TUTORIAL J U L Y 2 0 2 3 • M A T E R I A L S E V A L U A T I O N 43 2307 ME July dup.indd 43 6/19/23 3:41 PM

NDT/E, obtaining reliable labels is a spe- cialized and time-consuming task, which is further complicated as manual delin- eation of discontinuities introduces user subjectivity. In turn, mislabeled data can be counteractive to the learning process of supervised learning ML models (Taheri and Zafar 2023 Lever et al. 2017). Unsupervised learning: An ML paradigm from which an ML model is trained from the input data features, but without known data labels. Clustering is one of the most well-known forms of unsupervised learning, wherein data is divided into discrete groups. Furthermore, dimensionality reduc- tion and manifold learning methods such as principal component analysis (PCA) (Lever et al. 2017 Yang et al. 2022) and t-distributed stochastic neighbor embedding (tSNE) (van der Maaten and Hinton 2008) are forms of unsuper- vised learning. Unsupervised learning is useful in NDT/E due to the challenges of obtaining labels. Tips: If dependable labels can be obtained for a dataset, a supervised learning paradigm is often the simplest and most accurate. Assuming no labels are known, unsupervised learning is powerful but requires domain-specific insights from the user. Unsupervised learning also generally lacks metrics for standardized evaluation. Types of Learning Tasks Each ML paradigm can take on differ- ent tasks. In this subsection, we sub- divide supervised learning into its two most common tasks (classification and regression) and subdivide unsupervised learning into its two most common tasks (clustering and dimensionality reduc- tion). These subgroups are illustrated in Figure 1. Classification: A supervised ML model performs classification when it determines if the input data belongs to one of a discrete set of “classes,” or cat- egories. For example, different defect types (e.g., delamination, crack, no defect) may represent different classes that we may observe. Regression: A supervised ML machine model performs regression when estimating the value of a contin- uous dependent variable from an input independent variable. For example, an ML model may process imaging NDT/E data to estimate the size of a defect. Clustering: The clustering task aims to classify data without known informa- tion by identifying groups, or clusters, of data that are similar to each other in some manner. Clustering can be valuable for identifying unknown rela- tionships between the data, such as the presence of outlier data that could corre- spond to a discontinuity. Dimensionality reduction: The aim of dimensionality reduction is to reduce the data into its essential features. Many compression and denoising algorithms can be considered forms of dimensionality reduction (Yang et al. 2022). It can separate components (e.g., multiple reflections from an ultrasonic B-scan) that reconstruct the data when added together (Liu et al. 2015). This is sometimes referred to as blind source separation. Tips: It is important to determine the appropriate learning task for a given problem as it dictates the choice of an ML model and the associated challenges. Figure 1 describes the most common ML models used for each task. Characteristics of Machine Learning Datasets and Architectures Most ML architectures learn only from the provided data. As a result, ML model performance is highly depen- dent on the dataset quality. The classic bias-variance tradeoff is one of the most common challenges we must consider when building a dataset and choosing an architecture. Bias: One of the most significant issues that one must consider when creating a dataset is to consider the inherent bias that the dataset exhibits and how it affects the ML model. That is, a dataset will be biased if the training data (i.e., the input data and labels that are used to initially train the model) tends to better represent one scenario over another (Mehrabi et al. 2022). Note that bias is not inherently bad since you may want to focus on a particu- lar scenario (Miceli et al. 2022), but it is important to acknowledge that bias. For example, an ML model trained NDT TUTORIAL |AI/ML Labeled data Supervised learning Classification Regression Support vector machine Decision trees Random forests Linear regression Regularized regression Support vector regression Unlabeled data Unsupervised learning Clustering Dimensionality reduction K-means clustering Density-based clustering Principal component analysis Non-negative matrix factorization Figure 1. Learning paradigms of machine learning: (a) supervised learning algorithms utilize labeled data, which allows algorithms to be trained directly on the downstream task (classification and regression) (b) unsupervised algorithms utilize unlabeled data, which are primarily used for clustering and dimensionality reduction. Semi-supervised learning algorithms incorporate characteristics of both of these paradigms. 44 M A T E R I A L S E V A L U A T I O N • J U L Y 2 0 2 3 2307 ME July dup.indd 44 6/19/23 3:41 PM