J U L Y 2 0 2 0 • M A T E R I A L S E V A L U A T I O N 871 overview on algorithms for NDT classification is summa- rized in a previous paper (Aldrin and Lindgren 2018). Benefits of Algorithms/AI in NDT There are a number of advantages associated with incorpo- rating algorithms as part of an NDT technique. First, algo- rithms are typically very good at performing laborious and repetitive tasks. For most parts under test, either in manufac- turing or in service, the presence of critical NDT indications is a fairly rare event. Therefore, the data review process can often be a tedious task for most operators, who can expect mostly good parts. Second, given the amount and complexity of some data review tasks performed for some inspections, such tasks can be a challenge, especially for inexperienced inspectors or inspections that are rarely performed. This trend appears to be growing with the increasing quantity of data acquired with automated scanning and array sensing systems. Third, in many instances, algorithms can perform the data review task faster than manual review, providing potential savings in maintenance time and costs. Fourth, algorithms are typically not biased by expectation, such as the frequency of indications in past inspections. With a reduction in errors, the overall risk of maintaining a component can be improved. Fifth, algorithms can be designed in such a way to support the operator as a “digital assistant.” Algorithms could potentially help alleviate the burden of “mostly good data” and allow operators to focus on key data review tasks. As well, algo- rithms can be used to reduce the size and dimensionality of NDT data and present the operator with a reduced feature set for manual classification. Lastly, there are challenges with the aging workforce and transitioning expert knowledge to the next generation. Algorithms, if designed properly, can be repositories for expert knowledge in an NDT organization. Challenges of Algorithms/AI in NDT While the application of NDT algorithms shows great promise, there are a number of potential disadvantages with applying algorithm-based solutions to NDT inspection problems. First, the development and validation of reliable algorithms for NDT can be expensive. Training DLNNs requires very large, well-understood data sets, which are frequently not readily available for NDT applications. While the NDT community often possesses a large amount of data, the material state behind the data is often not perfectly known. Acquiring data from parts with well-characterized damage states, such as cracks, corrosion, or impact damage, requires either high-resolution NDT techniques for finger- printing, or destructive characterization for full verification. The design, training, and validation of algorithms also require unique software development skills and many hours of engi- neering labor to successfully implement. Second, algorithms also can perform poorly for scenarios that they are not trained to interpret. There have been concerns for decades about the reliability and adaptability of machine learning algorithms to completely perform complex NDT data review tasks. In NDT, many promising demonstra- tions have been performed by the NDT research community, but frequent issues concerning overtraining and robustness to variability for practical NDT measurements outside of the laboratory have been noted (Aldrin and Lindgren 2018). Prior successful NDT applications of neural networks have been dependent on taking great care to reduce the dimension- ality of the data and provide reliable features as inputs for clas- sification. As well, designing algorithms to address truly rare events—so-called black swans—is extremely difficult (Taleb 2007). Third, while human factors are frequently cited as being sources for error in NDT applications, humans are inherently more flexible in handling unexpected scenarios and can be better at making such judgement calls. Human inspectors also have certain characteristics like common sense and moral values, which can be beneficial in choosing the most reason- able and safest option. In many cases, humans can detect when an algorithm is making an extremely poor classification due to inadequate training and correct those errors. Fourth, for many machine learning algorithms like DLNNs, it can be difficult to ascertain exactly why certain poor calls are made. These algorithms are often referred to as “black boxes,” because the complex web of mathematical operations optimized for complex data interpretation problems does not generally lend itself to reverse engineering. Approaches are being developed to sample the parameters space to ascertain the likely source for decisions (Olden and Jackson 2002), but the field of “explainable AI” (XAI) is still in its infancy (Stapleton 2017). Lastly, with the greater reliance on algorithms, there is a concern about the degradation of inspector skills over time. As well, there is a potential for certain organizations to view automated systems and algorithms as a means of reducing the number of inspectors. However, many of these disadvantages can be mitigated through the proper design of human- machine interfaces. NDT Intelligence Augmentation With recent progress and hype on the coming wave of AI, some perspective is needed to understand how exactly these algorithms will be used by humans. While the original vision for AI was to mimic human intelligence, in practice AI has been successful only for very focused tasks. While today certain algorithms can perform better than humans for certain predefined and optimized tasks, we have not achieved the early goal of independent AI. Humans not only have the capa- bility to perform millions of different tasks, many in parallel run by the unconscious mind, but they also have the where- withal to determine when it is appropriate to switch between tasks and allow the conscious mind to have awareness as needed. The real value of AI today is using it as a specific tool (Aldrin et al. 2019).

872 M A T E R I A L S E V A L U A T I O N • J U L Y 2 0 2 0 As a counterpoint to AI, IA refers to the effective use of information technology to enhance human intelligence (Skagestad 1993 Rastogi 2017). This idea was proposed in the 1950s and 1960s by early cybernetics and computer pioneers. IA uses technology to essentially “support” a human in performing specific tasks. Relative to AI, IA has a long history of success. For example, consider the history of infor- mation technology, from the birth of writing and slide rules to smartphones and the internet. All of these forms of tech- nology have essentially been developed to extend the informa- tion storage and processing capabilities of the human mind. Fundamentally, progress on AI algorithms should be viewed as an evolution of tools to better support human performance. While most of the attention in recent years has been on the performance of AI over humans in games such as chess and Go (Lewis-Kraus 2016), there are a number of applica- tions that have been cited where humans plus algorithms can exceed the performance of computer algorithms alone. “Centaurs” (Scharre 2016 Case 2018) and “cyborgs” (Tharp 2017) are terms used to refer to such human-plus-machine collaborations. One example that is frequently cited is chess. A team of amateur chess players paired with three chess programs convincingly defeated a series of teams made up of chess grandmasters and some of the world’s best chess programs (Cowen 2013 Tharp 2017). While this case study is slightly dated and may not hold up to the success of AlphZero (Gerbert 2018), fundamentally, all of these algo- rithms at some stage in their design for operational tasks have incorporated human input. This collaboration between humans and algorithms leveraging high-performance computing has the potential to solve an array of greater problems than mere games of strategy. For example, for many decades the practice of engineering has consisted of humans leveraging their intellect with the support of computational tools to solve technical problems. Humans are still critical in asking the right questions and providing the appropriate focus, complementing the brute force computational power with creativity in selecting the most promising problem space to investigate (Wilson and Daugherty 2018). Humans also have a natural flexibility, versatility, and intuition that AI systems have yet to achieve. These uniquely human qualities are still quite impressive, especially considering the relatively low power consumption of the human mind. From the perspective of NDT applications incorporating algorithms, IA has the potential to address most of the disad- vantages of the AI-based algorithms cited previously. For example, many of the most promising DLNN applications today—from speech recognition to text translation and image classification—are still far from perfect. However, that does not mean that these tools are not useful. In practice, humans can frequently detect errors made by AI and can quickly work around poor results. Humans often develop an understanding where such algorithms can be most appropriately applied and where they should be avoided. By leveraging the algorithms where they are most useful, it becomes less critical for the algorithm to be able to handle all scenarios, especially very rare events. Lastly, by operators working in conjunction with algorithms, there is no need to pursue eliminating the human entirely. In general, the most cost-effective and reliable solution will mostly likely be some hybrid, human-plus-machine based approach. Human-Machine Interfaces Typical human interfaces with computer systems in NDT have included monitors, keyboards, mouses, and possibly joystick interactions. While these classic PC interfaces are still efficient for many tasks, there are also a number of emerging devices and tools that connect humans with automation. For example, industrial touchscreen tablets, augmented reality glasses, wearable devices (such as smart- watches), voice-recognition systems, and position tracking devices (such as Microsoft Kinect) all have the potential to provide more natural human-machine interfaces to support emerging NDT 4.0 systems. Several promising applications of augmented reality for aircraft maintenance applications have demonstrated feasibility in recent years (Avatar Partners 2017 Jordon 2018). Unique visualization support tools have also been developed for automatically aligning and visualizing data to 3D models, which enables detailed analysis to detect trends at specific locations on the model, indicating potential process problems (Sharp et al. 2009). Challenges for Implementation While this is an exciting time for new human-machine inter- face tools, there is a critical need to carefully optimize the fine interactions between humans and computer algorithms in NDT. Some work has studied the human-machine problem for different NDT applications (Dudenhoeffer et al. 2007 Bertović 2016a, 2016b). For example, Bertović performed a detailed survey of prior work on human factors when inter- facing with automation in NDT (Bertović 2016a). While extensive human-automation interaction has clear benefits, research suggests that increased automation has a number of challenges, costs (a paradox frequently dubbed as “automa- tion ironies” [Bainbridge 1987]), or “automation surprises” (Sarter et al. 1997). In this work, a failure modes and effects analysis (FMEA) was conducted to identify potential risks, and a number of preventive measures were proposed. Subse- quent studies were used to verify the benefit of the preventive measures, highlighting mixed levels of success (Bertović 2016a). Additional guidance on the challenge of human-machine interfaces can be gained from the experiences of other communities that also require very high levels of reliability. For example, in aviation, the use of autopilot systems and the handoff between human control and autopilot is a pertinent case study for NDT. In recent years, the accident rate for major aircraft has been reduced to one major accident per ME TECHNICAL PAPER w ia and human-machine interfaces