J U L Y 2 0 2 1 • M A T E R I A L S E V A L U A T I O N 745 position the robot such that the damage is accessible to the repair tool. The planning module will then instruct the tooling controller in execution of the repair. The tooling controller is a microcontroller that positions the repair actuator directly over the located damage. The repair actuator is an XYZ gantry system that uses three encoded linear motors for positioning. The encoder data are sent back to the tooling controller for accurate closed-loop control. The probe actuator keeps the NDE probe adhered to the material under test at the optimal distance, which results in more accurate NDE sensor readings. Lastly, the FSW tool that repairs the defect in the material under test is mounted at the end effector of the repairing actuator. When the NDE probe is in position, the tool is engaged and the repair is executed. After repair completion, the tool is retracted, and the robot picks up where it left off, navigating the boiler in search of more damage in need of repair. Defect Localization and Estimation Because certain infrastructure may have many defects, many of which are not critical to the boiler’s functionality, simple signal processing with a set threshold is not sufficient for damage classification. Thus, some form of machine learning approach is necessary. One unique challenge to address for the proposed CPS framework is that, in order to execute effective repairs, it is not only necessary to classify damage accurately but also feed damage locations into the global map of the robot’s environment through data fusion. To achieve this goal, simple classification approaches such as support vector machines (Noble 2006) or traditional convolutional neural networks (CNNs) (Albawi et al. 2017) are not adequate for this application because they are only able to classify whether an object is within an image, but have no awareness of where the object is within the image. To address this, an instance-based semantic segmentation method is utilized. This approach differs from traditional semantic segmentation in that it can detect the number of instances of a certain class instead of grouping them together as a single object. As this approach is concerned only with a single object class (the damaged area) inside the entire scan region and there may exist multiple cracks within the same scan, the proposed instance-based approach is ideal for this particular application. The authors’ approach utilizes a two-staged mask, region- based CNN (mask R-CNN) (He et al. 2017) as shown in Figure 7. First, data are fed into a traditional CNN for a cropped subsection, called an anchor, of each NDE scan image. To support training multiple images per batch, all images are resized such that the shortest edge is 800 pixels, but the aspect ratio of the original image is preserved. For each, or any anchor where crack instances are detected, a proposal of its bounding box is generated and stored in memory. The next stage merges each anchor containing the same instance of detected damage region as a single instance. Lastly, a pixel mask is generated for each predicted instance of the damage class. To train any neural network model, a sufficiently large training dataset is necessary, and for this application, NDE data of representative cracks for power plant boiler structures are also needed. A dataset with the number of required Input image Input image Exact region proposals Convert to grayscale Find bounding rectangle Feature extraction Classification Output Fully connected Convolution Pooling Crack? yes Scruff? no Scrape? no Generate pixel mask Figure 7. Mask R-CNN process, architecture, and pixel mask generation.

746 M A T E R I A L S E V A L U A T I O N • J U L Y 2 0 2 1 samples is currently not feasible to obtain. This would require NDE scans to be collected from hundreds of thousands of naturally occurring cracks in steel plates, which is too difficult and expensive to carry out at this time. Therefore, a transfer learning approach that builds off a preconstructed neural network architecture is utilized (Ma 2008). This model is a 101-layer CNN that has been pretrained on datasets containing over 15 million labeled images. Using this model allowed us to build off learned features from other data, elimi- nating the need to retrain an entire model. One hundred images of representative data instances were annotated and given to the model for training, which was implemented on eight graphics processing units (GPUs) for 160 000 iterations with a learning rate of 0.02, a weight decay of 0.0001, and momentum of 0.9. These parameters were chosen because, in addition to minimizing the chance of overfitting, they have been shown to provide an optimal balance between training time and premature convergence to a suboptimal solution. Once the model is trained and given an NDE scan containing a damaged region, the model will find the location of the damage and generate a pixel mask overlaid onto the scan image at the precise location of the damage. After the model has generated pixel masks over boiler damage instances, the location of each instance is calculated. The coordinates of a centroid are calculated by dividing the range of pixels that the masks occupy in horizontal and vertical directions by two. These image plane locations and the distance per pixel in the scan region correspond to where the damage exists on the boiler wall. With this information, the robot can then accurately position itself such that its repair probe is able to reach the region in need of repair. Steam corrosion, oxygen corrosion, alkali corrosion, and corrosion under scale are ever-present in the power plant boiler (Wu et al. 2019). The rough surfaces caused by the corrosion result in vibration during the movement of the robot. Therefore, the liftoff distance of the eddy current coil array may change due to this vibration during the inspection, which challenges the reliability of the NDE data. The eddy current coil array is placed with the liftoff distance in the range from 0 to 5 mm with a step size of 0.1 mm and 6 to 50 mm with a step size of 1 mm to inspect sample 2. As shown in Figure 8, all cracks have been successfully detected quantitatively, where the different normalized shifts of the resonance frequency of the eddy current sensor are correlated with the different depths of the narrow cracks. The normal- ized frequency shift decreases slightly with the increase of the liftoff distance of the eddy current sensor. According to the results, the crack detection capability of the sensor is basically ME TECHNICAL PAPER w ai-enabled robotic nde for structural damage 1 2 .1 .2 2 3 2 3 4 5 6 ( 7 8 9 10 11 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 1 18 19 20 21 22 23 24 25 L ( 26 2 28 29 30 31 32 33 34 35 6 37 38 40 41 2 44 45 47 8 9 0 2.4 2.5 2.6 Distance cm) 0 10 1 1 8 9 0 Liftoff distance (mm) 2 3 3 3 3 3 39 4 43 46 Figure 8. Normalized frequency shifts with different liftoff distance of the eddy current sensor. ×107 Normalized frequencyshifts