Geo-structures built on expansive clay, such as highway slopes in MS, are vulnerable to soil moisture variations and impacted by the increasing trend of high-intensity rainfall events. Early Identification of vulnerable or failing assets is crucial to strategize repairs and maintenance activities. Therefore, the first step of geotechnical asset management is to compile an inventory of the geostructures. Furthermore, frequent geo-structure health monitoring is crucial to avert risks and ensure asset longevity. Remote sensing-based infrastructure monitoring techniques have grown in popularity lately. However, identifying the vulnerable geostructures still requires boots on ground inspection. Such manual inspection after each weather event is expensive and impractical when quick action is needed. There is a need to automate the Geotechnical asset management methodology.To this end, this study used deep learning models integrated with ArcGIS Pro to detect and classify failed highway slopes in central MS. The failures studied included excessive deformations, shallow slides, sinkholes, etc. Training sample chips were first created by identifying failed slopes within Uncrewed Aerial Vehicle (UAV) based orthomosaic rasters and satellite imagery data. Comparative tests with different object detection and classification models with varying epochs were executed and documented. The results proved promising and assured capabilities of expanding the methodology to other infrastructure assets. The model can be run after each rainfall event to quickly spot any new slope instabilities. This innovative technology will ensure a quick inventory of failed and vulnerable geo-structures and help strategize maintenance and repair funds allocation. The geospatial deep learning methodology developed in this study is valuable for structural health monitoring and will help streamline Geotechnical Asset Management.