Earthquake Reaearch in China  2019, Vol. 33 Issue (1): 112-119     DOI: 10.19743/j.cnki.0891-4176.201901013
Information Detection of Seismic Debris Flow by UAV High-resolution Image Based on Transfer Learning
GUO Jiawei1,2, LI Yongshu1, WANG Hongshu3, LU Heng4,5, WANG Xiaobo6,7     
1. Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 611756, China;
2. Information Technology Center of Chengdu Planning and Management Bureau, Chengdu 610094, China;
3. Department of Surveying and Mapping Engineering, Sichuan Water Conservancy Vocational College, Chengdu 611231, China;
4. State Key Laboratory of Hydraulics and Mountain River Engineering, Sichuan University, Chengdu 610065, China;
5. College of Hydraulic and Hydroelectric Engineering, Sichuan University, Chengdu 610065, China;
6. Provincial Geomatics Center of Qinghai, Xining 810001, China;
7. Geomatics Technology and Application Key Laboratory of Qinghai Province, Xining 810001, China
Abstract: A large number of debris flow disasters (called Seismic debris flows) would occur after an earthquake, which can cause a great amount of damage. UAV low-altitude remote sensing technology has become a means of quickly obtaining disaster information as it has the advantage of convenience and timeliness, but the spectral information of the image is so scarce, making it difficult to accurately detect the information of earthquake debris flow disasters. Based on the above problems, a seismic debris flow detection method based on transfer learning (TL) mechanism is proposed. On the basis of the constructed seismic debris flow disaster database, the features acquired from the training of the convolutional neural network (CNN) are transferred to the disaster information detection of the seismic debris flow. The automatic detection of earthquake debris flow disaster information is then completed, and the results of object-oriented seismic debris flow disaster information detection are compared and analyzed with the detection results supported by transfer learning.
Key words: Earthquake     Debris flow     UAV high-resolution image     Transfer learning     Information detection    

INTRODUCTION

In 2008, the 5·12 Wenchuan Earthquake caused massive losses to the local people. For a long period of time after the earthquake, the seismic debris flows that often occurs during heavy rain brought about significant threats and injuries to people in the disaster area. According to statistics, the number of deaths caused by seismic landslides and debris flows during the Wenchuan earthquake accounted for about one-fourth of the total number of deaths from the earthquake (Yin Yueping, 2008). At the same time, the seismic debris flow can also cause many problems such as damage to buildings, traffic jams and barrier lakes. Therefore, how we use existing scientific methods to quickly detect seismic debris flow information and accurately obtain disaster information is one of the recent research priorities in earthquake disaster information detection and emergency rescues (Tang Chuan, 2010; Peng Shugang, 2014).

UAV remote sensing technology has become a common method in earthquake disaster information detection and emergency responses in the unique advantages of flexibility, speed, small impact by terrain or weather and high-resolution image (Ren Juan, 2015). However, the detection of seismic debris flow information based on UAV high-resolution images is mostly focused on visual interpretation, with low efficiency. Breaking through this technical bottleneck has become a hot research topic in this field.

With the continuous development of computer technology, deep learning has successfully made breakthroughs in automatic speech recognition, natural language processing and computer vision (Mikolov T. et al., 2013). Transfer learning is a kind of deep learning. It does not require the identically distributed hypothesis of training data and test data, which means that transfer learning does not need to calibrate a large amount of training data for each field, but rather transfer knowledge from existing data to help new learning tasks. Currently, feature-based transfer learning has broader transfer capabilities (Weiss K. et al., 2016; Saha B. et al., 2016). In view of this, based on the UAV high-resolution impact disaster database of seismic debris flow and using convolutional neural network (CNN), this paper conducts the training of seismic debris flow characteristics combined with the advantages of transfer learning features, in order to transfer the acquired training methods to the disaster information detection of seismic debris flow. Then, automatic detection of disaster information for seismic debris flow can be realized.

1 DATA SELECTION AND RESEARCH METHOD 1.1 Research Area and Data Profiling

The research area in this paper is Wenchuan County and Dujiangyan City, which are the worst-hit areas of the Wenchuan Earthquake. The geographical location is shown in Fig. 1. Wenchuan County is located in the northwestern edge of the Sichuan Basin, in the southeast region of the Aba Tibetan and Qiang Autonomous Prefecture, with a total area of 4, 083km2. Dujiangyan City is located to the west of Wenchuan County, on the northwestern edge of the Chengdu Plain, covering an area of 1, 208km2. The area was greatly affected by the earthquake which induced a large number of geological disasters such as mudslides, causing serious casualties and property damage.

Fig. 1 The geographical location of the study area

The black point in Fig. 1 is the earthquake-induced geological disaster, with the location of the high-resolution image of UAV in the area. The coverage area of the image is 76.59km2. Using the collected high-resolution images of UAV, samples of seismic debris flow disasters are collected, and then stored in a uniform format to form a sample database of seismic debris flow disasters, which includes 280 positive samples of seismic debris flow (seismic debris flow) and 1, 600 negative samples of seismic debris flow (non-seismic debris flows). All samples are scaled to 256×256 pixels. An example of seismic debris flow sample is shown in Fig. 2.

Fig. 2 Sample examples of high-grade image seismic debris flow of unmanned aerial vehicle
1.2 Technical Route and Method 1.2.1 Seismic Debris Flow Disaster Information Detection Based on the Object-oriented Classification Technology

In the field of remote sensing, the object-oriented classification technology is a classification method based on a target (object). It is an image classification technology developed for high-resolution image applications, which can make full use of the spectrum, texture, shape, spatial information, and adjacent relations of high-resolution remote sensing images to achieve segmentation of images and classification of objects. Thus, the classification results are closer to the results of visual interpretation, which can effectively improve the accuracy of classification. At present, the object-oriented classification technology has been widely used in remote sensing image classification. The object-oriented classification technology has two important features and key technical points, one is selecting the appropriate segmentation scale to segment the high-resolution image so that the detected features can be highlighted in the most appropriate segmentation scale, the other is selecting a variety of typical features of the segmentation object to establish a classification rule for the feature to detect or classify (Li Honghong, 2013).

The multi-scale segmentation of images is done in order to obtain a couple of objects by homogeneity criterion and heterogeneity criterion, and different segmentation parameter settings will result in different segmentation effects (Lu Heng et al., 2011). Fig. 3 shows the results of segmentation at different segmentation scales. Since the parameters of multi-scale segmentation shape and compactness both have little effect on the results, they are fixed and the influence of the scale parameters on the segmentation results will be analyzed. Seeing the effect of segmentation under multiple segmentation parameters, we can select the optimal segmentation parameters that are required to affect the classification task with flexibility, thereby improving the accuracy of classification.

Fig. 3 Effect diagram under different segmentation scales (a)The original UAV image.(b)Scale=120 shape=0.6 compact=0.5. (c)Scale=140 shape=0.6 compact=0.5.(d)Scale=160 shape=0.6 compact=0.5

After segmentation, how well the object of segmentation is can be judged by calculating the heterogeneity. The calculation formula is as follows:

$ f = w\cdot{h_{{\rm{color}}}} + \left({1 - w} \right)\cdot{h_{{\rm{shape}}}} $ (1)

In the formula: "f" stands for the heterogeneity of the total segmentation object; "w" is the weight of the defined shape parameter that is given by the user, the ranges of which are 0 to 1; Spectral heterogeneity is "hcolor", and heterogeneity of shape is "hshape" which consists of two parameters, namely the heterogeneity of tightness "hcompact "and the heterogeneity of smoothness "hsmooth".

The formula for calculating the shape heterogeneity is as follows:

$ {h_{{\rm{shape}}}} = {\rm{ }}{w_{{\rm{compaet}}}}\cdot{h_{{\rm{compact}}}} + (1 - {w_{{\rm{compact}}}})\cdot{\rm{ }}{h_{{\rm{smooth}}}} $ (2)

where, "wcompact" is the weight of tightness heterogeneity, ranges of which are 0-1.

The calculation formula for spectral heterogeneity is as follows:

$ {h_{{\rm{color}}}} = \sum\limits_c^{} {{w_c}} ({n_{{\rm{merge}}}}.\sigma _c^{{\rm{merge}}}({n_{{\rm{obj1}}}}.\sigma _c^{{\rm{obj1}}} + {n_{{\rm{obj2}}}}.\sigma _c^{{\rm{obj2}}})) $ (3)

where, "wc" stands of the band weights involved in splitting and merging; nmerge and σcmerge are symbol of combined area and its spectral variance; nobj1, σcobj1, nobj1, and σcobj1 are area of two adjacent regions and their spectral variance.

The formula for calculating the smoothness and compactness is as follows:

$ {h_{{\rm{smooth}}}} = {n_{{\rm{merge}}}}.\frac{{{l_{{\rm{merge}}}}}}{{{b_{{\rm{merge}}}}}} - ({n_{{\rm{obj1}}}}.\frac{{{l_{{\rm{obj1}}}}}}{{{b_{{\rm{obj1}}}}}} + {n_{{\rm{obj2}}}}.\frac{{{l_{{\rm{obj2}}}}}}{{{b_{{\rm{obj2}}}}}}) $ (4)
$ {h_{{\rm{compact}}}} = {n_{{\rm{merge}}}}.\frac{{{l_{{\rm{merge}}}}}}{{\sqrt {{n_{{\rm{merge}}}}} }} - ({n_{{\rm{obj1}}}}.\frac{{{l_{{\rm{obj1}}}}}}{{\sqrt {{{\rm{n}}_{{\rm{obj1}}}}} }} + {n_{{\rm{obj2}}}}.\frac{{{{\rm{l}}_{{\rm{obj2}}}}}}{{\sqrt {{{\rm{n}}_{{\rm{obj2}}}}} }}) $ (5)

where, lmerge, bmerge are the perimeter of the merged area and the perimeter of the outer rectangle, respectively. lobj1, bobj1, lobj2 and bobj2 stand for the perimeter of two adjacent regions and the perimeter of the outer rectangle.

According to the shape parameter and the spectral parameter, the segmentation is continuously adjusted, and the ideal segmentation object is obtained. Adjusting the scale parameter can indirectly do the same thing to the size of the image object, so a larger value of the parameter will obtain a larger object, and vice versa.

After the multi-scale segmentation of images, we can select the optimal segmentation parameters needed for information detection of seismic debris flow. By analyzing the features of high-resolution image segmentation objects, ones that can be used for information detection of seismic debris flow will be selected, and the custom characteristics of the detection is then established based on the feature information. We can then complete the information detection of seismic debris flow. The experimental images are divided into seismic debris flow areas and non-earthquake debris flow areas. To test the seismic debris flow areas, the formula for establishing the custom feature H is as follows:

$ H = 2 \times B - G - R $ (6)

where, R, G and B represent the average of the red, green and blue channels of the UAV image. When H belongs to the interval [30, 50] and the brightness of all bands is in the range of interval [150, 196], the information that is unclassified as the seismic debris flow disaster information can be identified as non-seismic debris flow information.

1.2.2 Information Detection of Seismic Debris Flow Disaster Based on Transfer Learning

The Convolutional Neural Network (CNN) is a multi-layer structure learning algorithm that improves training efficiency by analyzing spatial relative relationships (Zeiler M.D. et al., 2014). The CNN includes convolutional layers and downsampling layers. The convolutional layer is generated by convolution operation of the input image and a specific convolution template, and the downsampling layer is obtained by downsampling the feature map of the convolutional layer. The convolutional layer and the downsampling layer are alternately repeated, and with a number of fully connected layers, form a complete convolutional neural network.

By constructing a linear class elimination model of the convolutional neural network and eliminating linear fields such as roads and rivers, the integrity of the detected information of seismic debris flow can be maintained, and the interference of the linear features on the detection results can be reduced. The constructed linear class elimination model of convolutional neural network includes one input layer, three convolution layers, one FC256 fully connected layer and one FC2 output layer, as shown in Fig. 4.The output of the FC2 output layer in the figure is a linear feature type and a non-linear feature type, thereinto, the output of the non-linear feature result will be input as a parameter for information detection of seismic debris flow supported by transfer learning.

Fig. 4 Linear class elimination model of convolution neural network

It can be seen from the example sample of the seismic debris flow in Fig. 2 that the spatial shape and texture of the seismic debris flow varies greatly, and the spectral difference of the sample is large due to collection of multi-phase data. Taking advantage of the constructed sample database of the seismic debris flow disaster, and with the help of previous linear class eliminations of the model convolutional neural network, the support vector machine (SVM) classifier is used to remove the linear features in the model to transfer the information detection of seismic debris flow, which then constructs the detected information procedures of seismic debris flow, which is based on transfer learning.

The information detection of seismic debris flow disasters supported by transfer learning mainly includes feature learning, feature transference and training of seismic debris flow information detection model. For phase of feature learning, we use the linear class elimination model of the convolutional neural network as shown in Fig. 4 through the training sample database. While in the phase of feature transference, the parameters obtained by the training of the convolutional neural network are transferred, and the parameters of each layer of the model should be maintained, then an output layer is selected to output the characteristics of the seismic debris flow disasters. What's more, the feature vector of the debris flow from the phase of feature transference will be input into the support vector machine (SVM) classifier in the phase of model training.

Fig. 5 Transfer learning and detection of seismic debris flow disaster information flow
2 EXPERIMENTS AND ANALYSIS OF RESULTS 2.1 Results From Information Detection of Seismic Debris Flow

The high-resolution image of the UAV in Fig. 3(a) is selected as the experimental data. According to the object-oriented classification technique described in Section 1.2.1, the optimal segmentation scale of the high-resolution image of the UAV is 140, the shape parameter is 0.6 and the parameter of compactness is 0.5 through the repeated segmentation experiments. Finally, the object-based classification method is used to detect the disaster information of seismic debris flow in the area, the result of which is shown in Fig. 6(a). In the process of information detection of seismic debris flow disasters based on transfer learning from section 1.2.2, we need to analyze the activation results of the positive and negative sample input layers to attain the analysis of the training results of feature transference. The experiment found that the positive and negative samples of the seismic debris flow have better activation in the FC6 layer, which can be applied to distinguish between seismic debris flow and non-seismic debris flow. The results from information detection of seismic debris flow in this method are shown in Fig. 6(b).

Fig. 6 The results of seismic debris flow detected by two methods (a)The object-oriented classification method.(b)Information detection based on transfer learning
2.2 Analysis of Accuracy

Aiming at the above problem of seismic debris flow classification, the confusion matrix is used to evaluate the accuracy of the debris flow information detection by calculating the true positive rate (TPR), false positive rate (FPR) and accuracy (ACC).

It can be concluded from Table 1 that with the object-oriented classification method detecting the seismic debris flow information of this area, the TPR is 83.33%, FPR is 13.64%, and accuracy ACC is about 85.21%. For the seismic debris flow information detection based on transfer learning proposed in this paper, the FPR is 9.10%, the TPR is 79.63%, and the accuracy ACC is 86.62%.

Table 1 Accuracy evaluation system for seismic debris flow information detection

From the above parameters, the object-oriented classification method is superior to the information detection method based on transfer learning in true positive rate. However, the former is no better than the latter in the false positive rate. This tells us that the object-oriented classification method detects more non-target classes as target classes, which leads to higher accuracy of seismic debris flow information detection based on transfer learning than the object-oriented classification method.

3 CONCLUSIONS

The way automatically detected debris flow information is acquired from high-resolution images of UAV after an earthquake has become an issue in the current seismic emergency response environment. Combined with the convolutional neural network and transfer learning, this paper proposes an effective method for automatic detection of seismic debris flow information. The detection accuracy of this method is superior to the object-oriented classification method, and it is less likely to identify the target object as non-target objects. Nonetheless, this method also has the disadvantages of large sample demand, complicated training process and high computing resource requirements. Meanwhile, considering the huge potential of this method applied to the detection of spatial data information, the fusion of multi-source data will be applied to other high-resolution images for seismic information detection which includes building collapse and damage of road or bridge to provide reliable technical methods.

REFERENCES
Li Honghong. Study on Object Oriented Classification of High Resolution Remote Sensing Images and Information Extraction of Debris Flow[D]. Master's thesis. Nanjing: Nanjing University of Science and Technology, 2013 (in Chinese with English abstract).
Lu Heng, Li Yongshu, Tang Min. Evaluation and selection of segmentation scale with object-oriented method in multiple source remote sensing image of mountain areas[J]. Journal of Mountain Science, 2011, 29(6): 688–694 (in Chinese with English abstract).
Mikolov T., Chen Kai, Corrado G., Dean J. Efficient estimation of word representations in vector space[J]. Computer Science, 2013, arXiv: 1301.3781. https://arxiv.org/abs/1301.3781
Peng Shugang. Remote Sensing Extraction and Suspectibility Assessment of Earthquake-Triggered Debris Flows in the Wenchuan County[D]. Master's thesis. Xiangtan: Hunan University of Science and Technology, 2014 (in Chinese with English abstract).
Ren Juan. Debris Flow Hazard Assessment Based on UAV Remote Sensing and GIS[D]. Master's thesis. Chengdu: Chengdu University of Technology, 2015 (in Chinese with English abstract).
Saha B., Gupta S., Phung D., Venkatesh S.. Venkatesh S. Multiple task transfer learning with small sample sizes[J]. Knowledge and Information Systems, 2016, 46(2): 315–342. DOI:10.1007/s10115-015-0821-z.
Tang Chuan. Activity tendency prediction of rainfall induced landslides and debris flows in the Wenchuan earthquake areas[J]. Journal of Mountain Science, 2010, 28(3): 341–349 (in Chinese with English abstract).
Weiss K., Khoshgoftaar T.M., Wang Dingding. A survey of transfer learning[J]. Journal of Big Data, 2016, 3: 9. DOI:10.1186/s40537-016-0043-6.
Yin Yueping. Researches on the geo-hazards triggered by Wenchuan earthquake, Sichuan[J]. Journal of Engineering Geology, 2008, 16(4): 433–444 (in Chinese with English abstract).
Zeiler M.D., Fergus R. Visualizing and understanding convolutional networks. In: Proceedings of the 13th European Conference on Computer Vision[C]. Zurich, Switzerland: Springer, 2014: 818-833.