The destruction information of buildings is essential in earthquake damage assessment. In order to solve the problems of high labor cost and low efficiency in traditional building surface information identification,inspired by extracting building images based on deep learning,we propose a method combining UAV oblique photography model and deep learning to extract the destruction information on building surface after earthquake. In this paper,taking the Changning M_S6.0 earthquake as an example,we selected the slice map of the Shuanghe town oblique photographic model after earthquake as the data source. Then,we conducted a comparative analysis of the object-oriented classification,the network of VGG-16 and DeeplabV3+ for the extraction results of building surface damage information. Our results show that DeeplabV3+ has strong advantages for the extraction of building surface damage information,especially for the small cracks. The accuracy rate,recall rate,and overall precision of the method can reach 96.93%,96.85%,96.89% respectively,which can effectively extract building surface damage information. With more and more sample data accounted,the accuracy rate and recall rate will continue to increase,and the building surface information can be extracted more accurately,which has great practical application value.