872 M A T E R I A L S E V A L U A T I O N • J U L Y 2 0 2 0 As a counterpoint to AI, IA refers to the effective use of information technology to enhance human intelligence (Skagestad 1993 Rastogi 2017). This idea was proposed in the 1950s and 1960s by early cybernetics and computer pioneers. IA uses technology to essentially “support” a human in performing specific tasks. Relative to AI, IA has a long history of success. For example, consider the history of infor- mation technology, from the birth of writing and slide rules to smartphones and the internet. All of these forms of tech- nology have essentially been developed to extend the informa- tion storage and processing capabilities of the human mind. Fundamentally, progress on AI algorithms should be viewed as an evolution of tools to better support human performance. While most of the attention in recent years has been on the performance of AI over humans in games such as chess and Go (Lewis-Kraus 2016), there are a number of applica- tions that have been cited where humans plus algorithms can exceed the performance of computer algorithms alone. “Centaurs” (Scharre 2016 Case 2018) and “cyborgs” (Tharp 2017) are terms used to refer to such human-plus-machine collaborations. One example that is frequently cited is chess. A team of amateur chess players paired with three chess programs convincingly defeated a series of teams made up of chess grandmasters and some of the world’s best chess programs (Cowen 2013 Tharp 2017). While this case study is slightly dated and may not hold up to the success of AlphZero (Gerbert 2018), fundamentally, all of these algo- rithms at some stage in their design for operational tasks have incorporated human input. This collaboration between humans and algorithms leveraging high-performance computing has the potential to solve an array of greater problems than mere games of strategy. For example, for many decades the practice of engineering has consisted of humans leveraging their intellect with the support of computational tools to solve technical problems. Humans are still critical in asking the right questions and providing the appropriate focus, complementing the brute force computational power with creativity in selecting the most promising problem space to investigate (Wilson and Daugherty 2018). Humans also have a natural flexibility, versatility, and intuition that AI systems have yet to achieve. These uniquely human qualities are still quite impressive, especially considering the relatively low power consumption of the human mind. From the perspective of NDT applications incorporating algorithms, IA has the potential to address most of the disad- vantages of the AI-based algorithms cited previously. For example, many of the most promising DLNN applications today—from speech recognition to text translation and image classification—are still far from perfect. However, that does not mean that these tools are not useful. In practice, humans can frequently detect errors made by AI and can quickly work around poor results. Humans often develop an understanding where such algorithms can be most appropriately applied and where they should be avoided. By leveraging the algorithms where they are most useful, it becomes less critical for the algorithm to be able to handle all scenarios, especially very rare events. Lastly, by operators working in conjunction with algorithms, there is no need to pursue eliminating the human entirely. In general, the most cost-effective and reliable solution will mostly likely be some hybrid, human-plus-machine based approach. Human-Machine Interfaces Typical human interfaces with computer systems in NDT have included monitors, keyboards, mouses, and possibly joystick interactions. While these classic PC interfaces are still efficient for many tasks, there are also a number of emerging devices and tools that connect humans with automation. For example, industrial touchscreen tablets, augmented reality glasses, wearable devices (such as smart- watches), voice-recognition systems, and position tracking devices (such as Microsoft Kinect) all have the potential to provide more natural human-machine interfaces to support emerging NDT 4.0 systems. Several promising applications of augmented reality for aircraft maintenance applications have demonstrated feasibility in recent years (Avatar Partners 2017 Jordon 2018). Unique visualization support tools have also been developed for automatically aligning and visualizing data to 3D models, which enables detailed analysis to detect trends at specific locations on the model, indicating potential process problems (Sharp et al. 2009). Challenges for Implementation While this is an exciting time for new human-machine inter- face tools, there is a critical need to carefully optimize the fine interactions between humans and computer algorithms in NDT. Some work has studied the human-machine problem for different NDT applications (Dudenhoeffer et al. 2007 Bertović 2016a, 2016b). For example, Bertović performed a detailed survey of prior work on human factors when inter- facing with automation in NDT (Bertović 2016a). While extensive human-automation interaction has clear benefits, research suggests that increased automation has a number of challenges, costs (a paradox frequently dubbed as “automa- tion ironies” [Bainbridge 1987]), or “automation surprises” (Sarter et al. 1997). In this work, a failure modes and effects analysis (FMEA) was conducted to identify potential risks, and a number of preventive measures were proposed. Subse- quent studies were used to verify the benefit of the preventive measures, highlighting mixed levels of success (Bertović 2016a). Additional guidance on the challenge of human-machine interfaces can be gained from the experiences of other communities that also require very high levels of reliability. For example, in aviation, the use of autopilot systems and the handoff between human control and autopilot is a pertinent case study for NDT. In recent years, the accident rate for major aircraft has been reduced to one major accident per ME TECHNICAL PAPER w ia and human-machine interfaces

J U L Y 2 0 2 0 • M A T E R I A L S E V A L U A T I O N 873 2.56 million flights (Oliver et al. 2017). While overall air safety has been improving, incidents of the loss of control have not. Loss of control occurs when pilots fail to recognize and correct a potentially dangerous situation, causing an aircraft to enter an unstable condition. One example of the possible catastrophic consequences of automation is the tragic crash of flight AF447 (Oliver et al. 2017). Another very recent example concerns the Lion Air and Ethiopian Airlines crashes of Boeing 737 Max aircraft (Rice and Winter 2019). Such incidents are typically triggered by unexpected, unusual events, often comprising multiple conditions that rarely occur together, that fall outside of the normal repertoire of the pilot’s experience. In the case of the Boeing 737 Max, the handoff between the human and autopilot systems appears to have not been designed well, and many pilots were not instructed on its operation (Rice and Winter 2019). This paradox of almost totally safe systems, where the same tech- nology that allows the systems to be efficient and largely error-free, can also create systemic vulnerabilities that result in occasional catastrophes. Lessons learned from these cases include: (1) avoiding the cycle of implementing more automation to correct for poor human performance with existing autopilot systems (2) encouraging more hand-flying to prevent the erosion of basic piloting skills (3) improving the management of handovers from machines to humans (4) increasing pilot training for rare events and (5) supple- menting training using simulation of various rare event scenarios. It is critical to avoid the natural tendency to blame the human in these situations when the human-machine inter- face and/or algorithm design is poor (Hao 2019). Alterna- tively, it is important to find ways to make such systems robust and ideally “anti-fragile” to randomness and disorder in the environment (Taleb 2012). The human operator must have some level of “skin in the game” (Taleb 2012) and not become reliant on automation over time. Designing human- machine interfaces and providing the necessary training to achieve this balance is far from trivial. Best Practices for Design of IA and Human-Machine Interfaces Building on this prior work and experience, a series of best practices for IA in NDT 4.0 is proposed, highlighting how the operator should best interface with NDT data and algorithms. Algorithms clearly have a great potential to help alleviate the burden of big data in NDT however, it is important that operators are appropriately involved in secondary indication review and the detection of rare event conditions. The following best practices are proposed: l Provide inspectors with a natural user interface for NDT workflow management. Usability of human-machine inter- faces is a critical aspect of workflow management for NDT techniques, from setup, standardization, data acquisition, and indication review. Ideally, inspectors need a way to report results and efficiently provide feedback on indications. Frequently, there are means in NDT software systems to annotate indication results however, making this metadata readily available to external systems is one of the challenges for NDT 4.0 going forward. Such information will be very useful for refining NDT algorithms and improving life-cycle management. l Implement data analysis algorithms to address frequent NDT calls and complex data interpretation. It is important to address the low-hanging fruit on implementing algorithms for NDT applications and to help alleviate the burden for inspec- tors of reviewing “mostly good” data. As well, some complex interpretation problems (especially in ultrasonic NDT) can benefit from algorithms and data guides. The design of these algorithms requires a focus on the base capability for making NDT indication calls to provide value and help ensure relia- bility. The algorithm design process should consider the necessary engineering development time, cost for acquiring necessary data, and the approach with the highest likelihood of success. There will be a payoff for some applications, but not all applications may benefit from automation. However, as NDT 4.0 systems mature, development costs for each new application should be reduced. l Ensure inspectors provide a secondary review of indications and review data for rare events. While there is often an initial desire to have NDT algorithms make all indication calls and present simple (good or bad) calls, based on prior experi- ence, additional information is always requested by engi- neering and management to understand the details on why an indication call was made. Inspectors need a natural user interface to review each call with supporting data and provide feedback on the call details in light of the technical require- ments. As well, because no algorithm will be perfect, inspec- tors need to have a straightforward means to review NDT data quickly. This entails identifying rare indications and determining when the acquisition of the NDT data is out of specification. l Develop an integrated NDT “simulator” to provide operator training and support complex indication review. There is a potential to leverage the same software interface for training purposes, by having the operators periodically train and test their skills with various conditions in NDT data. Specific rare events can be stored and introduced periodically as part of the regular re-training process. Thus, the interface could be used similar to how flight simulators are used for pilots to verify their performance under standard conditions and rare events. As well, integrated models within the user interface can also provide a means for the verification of indications and support sizing by the inspectors. l Implement open architecture for NDT data and reporting. Promising software tools exist to support NDT practitioners with data archiving, visualization, and special queries (Sharp et al. 2009), and continued improvements with usability and functionality are expected in the future. Ideally, to share data between NDT 4.0 components, leveraging open data