Smart Informatix Laboratory

Smart Informatix Laboratory

Home Research Publications People Education Job Openings Contact


Structural Dynamic Response Estimation and System Identification Using Deep Convolutional Neural Networks


This study presents a deep convolution neural network (CNN) based approach to estimate the dynamic response of a linear single degree of freedom (SDOF) system, a nonlinear SDOF system, and a full-scale three-story multi-degree of freedom (MDOF) steel frame. In the MDOF system, roof acceleration is estimated through the input ground motion. Various cases of noise contaminated signals are considered in this study, and the conventionalmultilayer perceptron (MLP) algorithm serves as a reference for the proposed CNN approach. According to both the results from numerical simulations and experimental data, the proposed CNN approach is able to predict the structural responses accurately, and it is more robust against noisy data compared to the MLP algorithm. Moreover, the physical interpretation of CNN model is discussed in the context of structural dynamics. It is demonstrated that in some special cases the convolution kernel has the capability of approximating the numerical integration operator, and the convolution layers attempt to extract the dominant frequency signature observed in the ideal target signal while eliminating the irrelevant information during the training process.






Automated Defect Classification in Sewer Closed Circuit Television Inspections Using Deep Convolutional Neural Networks


Automated interpretation of sewer closed-circuit television (CCTV) inspection videos could improve the speed, accuracy, and consistency of sewer defect reporting. Previous research has attempted to use computer vision, namely feature extraction methods for automated classification of defects in sewer CCTV images. However, feature extraction methods use pre-engineered features for classifying images, leading to poor generalization capabilities. Due to large variations in sewer images arising from differing pipe diameters, in-situ conditions (e.g., fog and grease), etc., previous automated methods suffer from poor classification performance when applied to sewer CCTV videos. This paper presents a framework that uses an ensemble of binary deep convoluted neural networks (CNNs) to classify multiple defects in sewer CCTV images. A prototype system was developed to classify root intrusions, deposits, and cracks. The CNNs were trained and tested using 12,000 images collected from over 200 pipelines. The average testing accuracy, precision and recall were 86.2%, 87.7% and 90.6%, respectively, demonstrating the viability of this approach in the automated interpretation of sewer CCTV videos.






3D Dynamic Displacement-Field Measurement For Structural Health Monitoring Using Inexpensive RGB-D Based Sensor


This study presents a comprehensive experimental and computational study to evaluate the performance envelope of a representative RGB-D sensor (the first generation of Kinect sensor) with the aim of assessing its suitability for the class of problems encountered in the structural dynamics field, where reasonably accurate information of evolving displacement fields (as opposed to few discrete locations) that have simultaneous dynamic planar translational motion with significant rotational (torsional) components. This study investigated the influence of key system parameters of concern in selecting an appropriate sensor for such structural dynamic applications, such as amplitude range, spectral content of the dynamic displacements, location and orientation of sensors relative to target structure, fusing of measurements from multiple sensors, sensor noise effects, rolling-shutter effects, etc. The calibration results show that if the observed displacement field generates discrete (pixel) sensor measurements with sufficient resolution (observed displacements more than 10 mm) beyond the sensor noise floor, then the subject sensors can typically provide reasonable accuracy for transnational motion (about 5%) when the frequency range of the evolving field is within about 10 Hz. However, the expected error for torsional measurements is around 6% for static motion and 10% for dynamic rotation for measurements greater than 5°.






Convolutional Neural Network and Naive Bayes Date Fusion for Deep Learning-based Crack Detection


Corrosion is a major defect in structural systems that has a significant economic impact and can pose safety risks if left untended. Currently, an inspector visually assesses the condition of a structure to identify corrosion. This approach is time-consuming, tedious, and subjective. Robotic systems, such as unmanned aerial vehicles, paired with computer vision algorithms have the potential to perform autonomous damage detection that can significantly decrease inspection time and lead to more frequent and objective inspections. This study evaluates the use of convolutional neural networks (CNNs) for corrosion detection. A CNN learns the appropriate classification features that in traditional algorithms were hand-engineered. Eliminating the need for dependence on prior knowledge and human effort in designing features is a major advantage of CNNs. This study presents different CNN-based approaches for corrosion assessment on metallic surfaces.The effect of different color spaces, sliding window sizes, and CNN architectures are discussed. To this end, the performance of two pretrained state-of-the-art CNN architectures as well as two proposed CNN architectures are evaluated, and it is shown that CNNs outperform state-of-the-art vision-based corrosion detection approaches that are developed based on texture and color analysis using a simple multilayered perceptron network (MLP). Furthermore, it is shown that one of the proposed CNNs significantly improves the computational time in contrast with state-of-the-art pretrained CNNs while maintaining comparable performance for corrosion detection.






Evaluation of Convolutional Neural Networks for Deep Learning-based Corrosion Detection


Corrosion is a major defect in structural systems that has a significant economic impact and can pose safety risks if left untended. Currently, an inspector visually assesses the condition of a structure to identify corrosion. This approach is time-consuming, tedious, and subjective. Robotic systems, such as unmanned aerial vehicles, paired with computer vision algorithms have the potential to perform autonomous damage detection that can significantly decrease inspection time and lead to more frequent and objective inspections. This study evaluates the use of convolutional neural networks (CNNs) for corrosion detection. A CNN learns the appropriate classification features that in traditional algorithms were hand-engineered. Eliminating the need for dependence on prior knowledge and human effort in designing features is a major advantage of CNNs. This study presents different CNN-based approaches for corrosion assessment on metallic surfaces.The effect of different color spaces, sliding window sizes, and CNN architectures are discussed. To this end, the performance of two pretrained state-of-the-art CNN architectures as well as two proposed CNN architectures are evaluated, and it is shown that CNNs outperform state-of-the-art vision-based corrosion detection approaches that are developed based on texture and color analysis using a simple multilayered perceptron network (MLP). Furthermore, it is shown that one of the proposed CNNs significantly improves the computational time in contrast with state-of-the-art pretrained CNNs while maintaining comparable performance for corrosion detection.






Review of Reconfigurable Swarm Robots for Structural Health Monitoring


Recent advancements in robotic systems have led to the development of reconfigurable swarm robots (RSR) that can change their shape and functionality dynamically, without any external intervention. RSR have the advantages of being modular, on-site reconfigurable, multifunctional, incrementally assemble-able, reusable, fault-tolerant, and even repairable on the orbit. Newly-developed reconfigurable robots are expected to bring a radical change in the prevailing structural health monitoring techniques, thus augmenting the efficiency, accuracy and affordability of inspection operations. This study presents a holistic review of the previous studies and state-of-the-art technologies in the field of RSR, and argues that RSR offer great potential advantages from the perspective of monitoring and assessment of civil and mechanical systems. A roadmap for future research has also been outlined based on the limitations of the current methods and anticipated needs of future inspection systems.






Robust 3D Scene Reconstruction in the Presence of Misassociated Features


Georeferencing through aerial imagery has several applications, including remote sensing, real-time situational mission awareness, environmental monitoring, rescue and relief, map generation, and autonomous hazard avoidance, landing and navigation of Unmanned Aerial Vehicles (UAV). In aerial imagery, Structure from Motion (SfM) is often used for 3D point reconstruction (i.e., ground locations) and for camera pose estimation (i.e., airborne position and orientation) from a set of geometrically matched features between 2D images. We introduce an adaptive resection-intersection bundle adjustment approach that refines the 3D points and camera poses separately after the "gross" misassociations are removed by an outlier rejection algorithm. For each iteration, the proposed approach identifies the potential misassociated features independently in the resection as well as the intersection stage, where these potential outliers, contrary to previous studies, are reexamined at later iterations. In this way, maximum number of inlier matched features is retained.






Color and Depth Data Fusion for Dynamic Displacement-Field Measurement


While there are several sensors for direct displacement measurements at a specific point in a uniaxial direction or multi-component deformations, there are only very limited, and relatively quite expensive, methodologies for obtaining the three-dimensional components of a displacement of a dynamically evolving (i.e., not pseudo-statically) deformation field. This study reports the results of a comprehensive experimental study to assess the accuracy and performance of a class of inexpensive vision-based sensors (i.e., RGB-D sensors) to acquire dynamic measurements of the displacement field of a test structure. It is shown that the class of sensors under discussion, when operated under the performance envelope discussed in this paper, can provide, with acceptable accuracy, a very convenient and simple means of quantifying three-dimensional displacement fields that are dynamically changing at relatively low-frequency rates typically encountered in the structural dynamics field.






Texture-based Video Processing Using Bayesian Data Fusion for Crack Detection


Regular inspection of the components of nuclear power plants is important to improve their resilience. Prevalent automatic crack detection algorithms may not detect cracks in metallic surfaces because these are typically very small and have low contrast. Moreover, the existence of scratches, welds, and grind marks leads to a large number of false positives when state-of-the-art vision-based crack detection algorithms are used. In this study, a novel crack detection approach is proposed based on local binary patterns (LBP), support vector machine (SVM), and Bayesian decision theory. The proposed method aggregates the information obtained from different video frames to enhance the robustness and reliability of detection.






Multimodal Sensor Fusion for Autonomous Data Acquisition of Road Surfaces


In this study, the development, evaluation, calibration, and field application of a novel, relatively inexpensive, vision-based sensor system employing commercially available off-the-shelf devices, for enabling the autonomous data acquisition of road surface conditions are performed. It is shown that the proposed multi-sensor system, by capitalizing on powerful data-fusion approaches of the type developed in this study, can provide a robust cost-effective road surface monitoring system with sufficient accuracy to satisfy typical maintenance needs, in regard to the detection, localization and quantification of potholes and similar qualitative deterioration features where the measurements are acquired via a vehicle moving at normal speeds on typical city streets. The proposed system is ideal to be used for crowdsourcing where several vehicles would be equipped with this cost-effective system for more frequent data collection of road surfaces.






Microcrack Assessment on Reactor Internal Components of Nuclear Power Plants


Ageing power facilities are increasingly susceptible to the onset of damage related to long exposure to stress, radiation, elevated temperatures and environmental conditions. One failure mechanism of particular concern is the onset of stress corrosion cracking. Currently, a technician manually measures the crack thicknesses at few points along a microcrack in a microscopic image, and the results are quantified by the Root Mean Square (RMS) of these measurements. In this study, a vision-based methodology is proposed for accurate quantification of microcracks that provides the thickness measurements for each pixel along the crack centreline and provides more comprehensive insight regarding the condition of a microcrack. A region-growing method is used for segmenting microcracks from complex backgrounds. The microcrack thicknesses are then automatically computed along the lines orthogonal to the crack centreline. The fast marching method is used to accurately estimate the centreline of microcracks.






Enhanced Crack Width Measurement


In this study a new contact-less crack quantification methodology, based on computer vision and image processing concepts, is introduced. In the proposed approach, a segmented crack is correlated with a set of proposed strip kernels. For each centerline pixel, the minimum computed correlation value is divided by the width of the corresponding strip kernel to compute the effective crack thickness. In order to obtain more accurate results, an algorithm is proposed to compensate for perspective errors.






Unsupervised Defect Detection for Autonomous Pavement Condition Assessment


Current pavement condition assessment procedures are extensively time consuming and laborious; in addition, these approaches pose safety threats to the personnel involved in the process. In this study, an RGB-D sensor is used to detect and quantify defects in pavements. This sensor system consists of an RGB color image, and an infrared projector and camera which act as a depth sensor. An approach, which does not need any training, is proposed to interpret the data sensed by this inexpensive sensor. This system has the potential to be used for autonomous cost-effective condition assessment of road surfaces. Various road conditions including patching, cracks, and potholes are autonomously detected and, most importantly, quantified, using the proposed approach. Several field experiments have been carried out to evaluate the capabilities, as well as the limitations of the proposed system. GPS information is incorporated with the proposed system to localize the detected defects.






Color and Texture Analysis for Corrosion Detection


Corrosion is a crucial defect in structural systems that can lead to catastrophic effects if neglected. In this study, we have evaluated several parameters that can affect the performance of color wavelet-based texture analysis algorithms for detecting corrosion. Furthermore, an approach is proposed to utilize the depth perception for corrosion detection. The proposed approach improves the reliability of the corrosion detection algorithm.The integration of depth perception with pattern classification algorithms, which has never been reported in published studies, is part of the contribution of this study. Several quantitative evaluations are performed to scrutinize the performance of the investigated approaches.






Contactless Crack Quantification


We developed vision-based crack quantification approaches that utilize depth perception to quantify crack thickness and, as opposed to most previous studies, need no scale attachment to the region under inspection, which makes these approached ideal for incorporation with autonomous or semi-autonomous mobile inspection systems including unmanned aerial vehicles. Guidelines are presented for optimizing the acquisition and processing of images, thereby enhancing the quality and reliability of the damage detection approach and allowing the capture of even the slightest cracks (e.g., detection of 0.1 mm cracks from a distance of 20 m), which are routinely encountered in realistic field applications where the camera-object distance is not controllable.






Crack Detection through Incorporation of Depth Perception


Inspired by human vision, where depth perception allows a person to estimate an object's size based on its distance to the object, we have introduced the integration of depth perception into image-based crack detection algorithms. This approach has improved the performances of crack detection and quantification systems. Whereas other proposed crack detection techniques used fixed parameters, this system utilizes depth perception to detect cracks. To this end, the crack segmentation parameters are adjusted automatically based on depth parameters. This feature is more practical for field applications where the camera-object distance cannot be controlled such as when unmanned aerial vehicles are used for data collection. The depth perception is obtained using 3D scene reconstruction.






Multi-Image Stitching for Evaluating Change Evolution and Inspection


This study presents and evaluates the underlying technical elements for the development of an integrated inspection software tool that is based on the use of inexpensive digital cameras. To this end, digital cameras are appropriately mounted on a structure (e.g., a bridge) and can zoom or rotate in three directions (similar to traffic cameras). They are remotely controlled by an inspector, which allows the visual assessment of the structure's condition by looking at images captured by the cameras. The proposed system gives an inspector the ability to compare the current (visual) situation of a structure with its former condition. If an inspector notices a defect in the current view, he/she can request a reconstruction of the same view using images that were previously captured and automatically stored in a database. Furthermore, by generating databases that consist of periodically captured images of a structure, the proposed system allows an inspector to evaluate the evolution of changes by simultaneously comparing the structure's condition at different time periods.





Copyright © 2014-2018 Smart Informatix Laboratory, Purdue University. All rights reserved.