Counterfactual explanations are an emerging tool to enhance interpretability of deep learning models. Given a sample, these methods seek to find and display to the user similar samples across the decision boundary.In this paper, we propose a generative adversarial counterfactual approach for satellite image time series in a multi-class setting for the land cover classification task. One of the distinctive features of the proposed approach is the lack of prior assumption on the targeted class for a given counterfactual explanation. This inherent flexibility allows for the discovery of interesting information on the relationship between land cover classes. The other feature consists of encouraging the counterfactual to differ from the original sample only in a small and compact temporal segment. These time-contiguous perturbations allow for a much sparser and, thus, interpretable solution. Furthermore, plausibility/realism of the generated counterfactual explanations is enforced via the proposed adversarial learning strategy.

Bioacoustics, the study of animal vocalizations and natural soundscapes, has proven to be a valuable source of data for wildlife monitoring. Just as a human would use contextual information to identify species calls from acoustic recordings, one unexplored way to improve deep learning classifier in bioacoustics is to provide the algorithm with contextual meta-data, such as time and location. We developed an algorithm to classify 22 bird songs for which the location can help to distinguish the different species. We explored different multi-branch convolutional neural networks, trained on both spectrograms and location information, as well as a geographical prior separately trained on location to estimate the probability that a species occurs at a given location. We compared the classification of the models to a baseline model without the spatial meta-data. Our findings revealed in each case an increase in the performance of the classification with the highest improvement obtained with the geographical prior (F1-score of 87.78\%, compared to 61.02\% for the baseline model). The methods based on multi-branch neural network proved to be efficient as well and simpler to use than the geographical prior as it requires a single model. Adding metadata to the acoustic classifier is a valuable source of information to improve classification performance, with room for further progress, and opens new opportunities for generalizing models.

Jointly harnessing complementary features of multi-modal input data in a commonlatent space has been found to be beneficial long ago. However, the influence ofeach modality on the model’s decision remains a puzzle. This study proposesa deep learning framework for the modality-level interpretation of multimodalearth observation data in an end-to-end fashion. While leveraging an explainablemachine learning method, namely Occlusion Sensitivity, the proposed frameworkinvestigates the influence of modalities under an early-fusion scenario in whichthe modalities are fused before the learning process. We show that the task ofwilderness mapping largely benefits from auxiliary data such as land cover andnight time light data.

In this paper we propose a mask-conditional synthetic image generation model for creating synthetic satellite imagery datasets. Given a dataset of real high-resolution images and accompanying land cover masks, we show that it is possible to train an upstream conditional synthetic imagery generator, use that generator to create synthetic imagery with the land cover masks, then train a downstream model on the synthetic imagery and land cover masks that achieves similar test set performance to a model that was trained with the real imagery. Further, we find that incorporating a mixture of real and synthetic imagery acts as a data augmentation method, producing better models than using only real imagery (0.5834 vs. 0.5235 mIoU). Finally, we find that encouraging diversity of outputs in the upstream model is a necessary component for improved downstream task performance. We have released code for reproducing our work on GitHub: redacted for double-blind peer review.

Protecting and restoring ecological connectivity is essential to climate change adaptation, and necessary if species are to shift their geographic distributions to track their suitable climatic conditions over the coming century. Despite the increasing availability of near real-time and high resolution data for landcover change, current connectivity planning projects are hindered by the computational time required to run connectivity analyses at realistic geographic scales with realistic models of movement. This bottleneck precludes application of optimization algorithms to prioritize ecological restoration to maintain and improve connectivity. Here we propose we can make progress toward overcoming these challenges using machine-learning methods. Our proposed methods will enable rapid optimization of connectivity prioritization and extend its application to many more species than is currently possible. We conclude by illustrating how this project will contribute to efforts to apply connectivity conservation using an example of ongoing restoration in southern Québec.

Poverty maps derived from satellite imagery are increasingly used to inform high-stakes policy decisions, such as the targeting of humanitarian aid and the allocation of government resources. These maps are typically constructed by training machine learning algorithms with country- or continent-scale data, but many real-world applications are focused on specific urban or rural areas. This paper shows that satellite-based poverty predictions are less accurate at distinguishing levels of wealth within urban and rural areas than they are at distinguishing wealth differences between urban and rural areas, investigates why this may be the case, and documents the implications of these disparities for downstream policy decisions.

Recent research in demography focuses on linking population data to environmental indicators. Satellite imagery can support such projects by providing data at a large scale and a high frequency. Moreover, population surveys often provide geolocations of households, yet sometimes with an offset, to guarantee data confidentiality. In such cases, the proper management of this incertitude is required, to accurately link environmental indicators such as land cover/land use maps or spectral indices to population data. In this paper, we introduce a method based on the random sampling of possible households geolocations around the coordinates provided. Then, we link a land cover map generated using semi-supervised deep learning and a Malaria Indicator Survey in Burkina Faso. After linking households to their close environment, we distinguish several types of environment conducive to high malaria rates, beyond the urban/rural dichotomy.

In geospatial planning, it is often essential to represent objects in a vectorized format, as this format easily translates to downstream tasks such as web development, graphics, or design. While these problems are frequently addressed using semantic segmentation, which requires additional post-processing to vectorize objects in a non-trivial way, we present an Image-to-Sequence model that allows for direct shape inference and is ready for vector-based workflows out of the box. We demonstrate the model's performance in various ways, including perturbations to the image input that correspond to variations or artifacts commonly encountered in remote sensing applications. Our model outperforms prior works when using ground truth bounding boxes (one object per image), achieving the lowest maximum tangent angle error.

This panel will feature a discussion between experts on topics related to the challenges of bridging the gap between research and deployment of ML methods for remote sensing data, including: - Experiences working with interdisciplinary teams, and leveraging the different expertises being brought to the table by different people - Thinking about end users / stakeholders and designing products which are accessible to them - Considerations when deploying Machine Learning for Remote Sensing projects at global scales

Climate-induced disasters are and will continue to be on the rise, and thus search-and-rescue (SAR) operations, where the task is to localize and assist one or several people who are missing, become increasingly relevant. In many cases the rough location may be known and a UAV can be deployed to explore a confined area to precisely localize the people. Due to time and battery constraints it is often critical that localization is performed efficiently. We abstract this type of problem in a framework that emulates a SAR-like setup without requiring access to actual UAVs. In this framework, an agent operates on top of an aerial image (proxy for a search area) and must localize a goal that is described through visual cues. To further mimic the situation on a UAV, the agent cannot observe the search area in its entirety, not even at low resolution, so it must operate based on partial glimpses alone. To tackle this task, we propose AiRLoc, a reinforcement learning (RL) model that decouples exploration (searching for distant goals) and exploitation (localizing nearby goals). Extensive evaluations show that AiRLoc outperforms various baselines as well as humans, and that it generalizes across datasets, e.g. to disaster-hit areas without seeing a single disaster scenario during training. Code and models are available at https://github.com/aleksispi/airloc.

Plastic litter is a major environmental hazard that endangers human, animal, and plant health on the planet. A substantial portion of plastic pollutants is washed from rivers and beaches into the oceans and aggregates at the surface as marine debris before decomposing into microplastics and being digested by animals or sedimented on the sea floor. The marine debris is inherently difficult to annotate manually on satellite images, as the boundaries of floating objects are not sharp and a certain mixture of water is always present at the pixel level. Hence, all available annotated marine debris datasets suffer from annotation errors. In this work, we present a label refinement algorithm for marine debris detection that improves upon rough hand annotations and takes the spectral characteristics of marine debris into account. We show quantitatively that a deep learning model trained with improved annotations achieves a higher classification accuracy on confirmed marine debris on two out of three datasets of confirmed plastic marine debris in Africa (in Ghana and South Africa). Thanks to the refinement module, we improve results for an environmentally important application that would benefit from further research attention to mitigate important associated challenges like label noise, domain shifts, and severe class imbalance.

Despite widespread adoption of deep learning models to address a variety of computational vision tasks, planetary science has yet to see extensive utilization of such tools to address its unique problems. On Titan, a moon of Saturn, tracking seasonal trends and weather patterns of clouds provides crucial insights into one of the most complex climates in the Solar System, yet much of the available image data is still processed manually. We demonstrate that transfer learning techniques can deliver a high degree of accuracy for cloud detection on the data collected from the Cassini–Huygens Mission to Saturn from 1997-2017. We present this work to encourage others to join us in analysis of cloud data throughout the Solar System, as future telescopy projects promise an influx of images in the coming years.

As geospatial machine learning models and maps derived from their predictions are increasingly used for downstream analyses in science and policy, it is imperative to evaluate their accuracy and applicability. Geospatial machine learning has key distinctions from other learning paradigms, and as such, the correct way to measure performance of spatial machine learning outputs has been a topic of debate. In this paper, I delineate unique challenges of model evaluation for geospatial machine learning with global or remotely sensed datasets, culminating in concrete takeaways to improve evaluations of geospatial model performance.

The communication between ground stations and low earth orbit satellites is limited by a window of time as well as by the signal transmission speed. As a consequence, machine learning models for remote sensing need to be reasonably small in order to be transmitted and loaded to the device. Top performing deep learning models in the literature usually include millions of parameters, which limits their potential use on board once the satellite is in orbit. This paper is inspired by a previous work, PRANC, which explores the feasiblity of using a linear combination of multiple pseudo-randomly generated frozen models for classification purposes. We extend its use to semantic segmentation of building footprints. While this is not a reduction technique as such, results demonstrate that these type of models can be easily transmitted and reconstructed on board without compromising the model performance. In particular, the network reaches a competitive performance, while requiring only hundreds of kilobytes.

Deep learning-based methods are state-of-the-art methods for the semantic segmentation of dwellings. However, their performance can severely drop when they are used outside of the trained domain, which is often the case for rapid segmentation tasks that appear as a consequence of forced population displacement in cases of disasters, conflicts or political instabilities. Unsupervised Domain Adaptation has been proposed as a possible solution for such an issue as it tries to adapt a classifier trained on a specified domain with labels to help predict in a different domain without labels. Inspired by recent success of optimal transport in the context of domain adaptation, we propose a new unsupervised domain adaptation technique for semantic segmentation (SegJUMBOT). This method addresses the domain shift problem by leveraging the unbalanced minibatch-based optimal transport framework in the case of semantic segmentation of large remote sensing datasets. We apply our novel methodology to a challenging adaptation problem where we leverage a standard building detection dataset (INRIA Aerial Image Labelling Dataset) acquired over European cities to detect footprint of buildings in the Bangladesh site of Kutupalong. In our experiments, our method compares positively to other optimal transport based methods.

The ability to track and process large images and focus only on areas of interestis extremely important to reduce computational resources, which translates toa decrease in costs, when working on the cloud, or energy, when working withedge computing devices. A clear remote sensing task that can benefit from suchreduction consists in detecting ships in open seas. In particular, in such scenariothousands of miles may only contain water without ships. In this work, wepropose an efficient cascade architecture that could be deployed in any machine.To achieve our goal, current state-of-the-art image-based machine learningalgorithms will be assessed, building on top of previous related work.

Understanding and properly estimating the impacts of environmental interventions is of critical importance as we work towards achieving global climate goals. Advances in machine learning paired with the growth of accessible satellite imagery have led to increased utilization of remotely sensed measures when inferring the impact of a policy. However, when machine learning models simply minimize a standard loss function, the predictions that they generate can produce biased estimates in downstream causal inference. If prediction error in the outcome variable is correlated with policy variables or important confounders, as is the case for many widely used remote sensing data sets, estimates of the causal impacts of policies can be biased. In this paper, we demonstrate how this bias can arise, and we propose the use of an adversarial debiasing model (Zhang, Lemoine, and Mitchell 2018) in order to correct the issue when using satellite datato generate machine learning predictions for use in causal inference. We apply this method to a case study of the relationship between roads and tree cover in West Africa, where our results indicate that adversarial debiasing can recover a much more accurate estimate of the parameter of interest compared to when the standard approach is used.

Accurate yield forecasting is essential for making informed policies and long-termdecisions for food security. Earth Observation (EO) data and machine learning al-algorithms play a key role in providing a comprehensive and timely view of cropconditions from field to national scales. However, machine learning algorithms'prediction accuracy is often harmed by spatial heterogeneity caused by exogenous factors not reflected in remote sensing data, such as differences in crop management strategies. In this paper, we propose and investigate a simple technique called state-wise additive bias to explicitly address the cross-region yield heterogeneity in Kazakhstan. Compared to baseline machine learning models (Random Forest, CatBoost, XGBoost), our method reduces the overall RMSE by 8.9% and the highest state-wise RMSE by 28.37%. The effectiveness of state-wise additive bias indicates machine learning’s performance can be significantly improved by explicitly addressing the spatial heterogeneity, motivating future work on spatial-aware machine learning algorithms for yield forecasts as well as for general geospatial forecasting problems.

This paper elaborates a new change detection algorithm, the Pixel-Wise T-Test (PWTT), developed to leverage the particular characteristics of war-induced building damage. The algorithm is deployed and tested in several cities damaged during the war in Ukraine. Despite being simple and lightweight, the algorithm produces results with accuracy statistics rival State of the Art methods that use deep learning and expensive high resolution imagery. Furthermore, the workflow is deployed entirely within the Google Earth Engine environment, allowing for the generation of near-real time damage maps that allow humanitarian practitioners to immediately get the count of damaged buildings in a user-specific area of interest.