Project institution: Lancaster University
Project supervisor(s): Prof Plamen Angelov (Lancaster University), Prof Peter M. Atkinson (Lancaster University), Prof Ana Basiri (University of Glasgow)
Research area: Geodynamics, Geoscience and Environmental Change
Industry partner: Microsoft
xLEO offers a paradigm shift from static, periodically retrained deep learning (DL) models towards lifelong learning grounded in interpretable prototypes. Sentinel satellites which are used for Earth observation (EO) deliver petabytes of continuous stream of multi-modal data. Yet, DL models are typically trained once on a fixed dataset, making them prone to catastrophic forgetting when adapting to new data. Retraining from scratch is infeasible, while opaque model weights limit user trust in high-stakes monitoring of changes and, particularly, fast changes (e.g., floods). xLEO aims to investigate how knowledge can be represented through prototypes, preserved as data patterns drift, and updated when new observations arrive. It will also explore how information from different modalities and physics-informed knowledge about the environment can support more reliable learning. GPU-based computing will support the processing of large EO datasets and future scaling of the approach.
Methods Used:
xLEO builds on prototype-based continual learning. Instead of relying on knowledge stored in millions of opaque weights, xLEO represents knowledge through prototypes, which are representative, real observations drawn from the data. Each incoming image is compared to these prototypes, allowing every decision by the model to be traced back to real examples. In this prototype-based learning, prototypes and a small set of support samples are stored in a replay buffer to prevent catastrophic forgetting. A maximum mean discrepancy (MMD) cluster preservation loss keeps the learned representation stable as new data arrive, so that old knowledge is retained while the model adapts to new knowledge. This approach builds on the label-free replay buffer and cluster preservation method (Aghasanli et al., CVPR 2025), which has been demonstrated on benchmark datasets, but, yet to be applied to Earth observation. The two teaser projects share the prototype-based continual learning foundation but explore different aspects of the problem. The first focuses on continual learning for flood monitoring as data patterns change and the second focuses on learning from multimodal environmental data streams and combining the knowledge representations they produce. Together they lay the groundwork for physics-informed DL.
Teaser Project 1 Objectives: Continual learning for flood monitoring
Flood observations vary across geography, seasons and climatic conditions. A model which is trained on an initial set of observations struggles when new flood event occurs or when incoming data is out of distribution. Adapting to these new observations causes the model to forget the previously acquired knowledge. This six-month project will investigate prototype-based continual learning as an instrument to provide explainable yet highly accurate solutions and, moreover, to retain the previously acquired knowledge while adapting to changes.
The key objectives are as follows:
- Establishing a baseline with flooding related observations from the RaVÆn dataset and developing a suitable model.
- Sequentially introduce observations across various flood events and evaluate the model’s ability to adapt to changing data patterns while preserving the previously acquired knowledge.
- Assess the extent to which the prototype-based approach and the new cluster preservation functions can reduce catastrophic forgetting as the incoming observations change.
Teaser Project 2 Objectives: Continual learning for environmental monitoring from multimodal data streams
Environmental monitoring often relies on a multitude of data sources in addition to the satellite imagery. Topography, environmental measurements and information from physical models often provide a richer context that complements the data obtained from the satellite. Combining these sources requires addressing a number of questions, including fusion of latent space representation derived from the deep learning models with physics-based interpretable features. While deep learning models offer unparalleled accuracy their interpretation is still an open research question. Combining interpretable features such as the ones used in physics-based models with deep learning representations is a challenging question. This project offers to step upon knowledge representation via prototypes which are interpretable and fusion with physics-derived constraints. As a result, the multimodal models will learn and maintain meaningful relationships between the knowledge representations which are obtained from different modalities. These relationships may change as environmental conditions and incoming data patterns change, hence the model needs to adapt while retaining useful observations. This six-month project will investigate continual learning from multimodal environmental data. It will focus on satellite imagery and one complementary environmental data source, selected according to the task and data availability.
- Define a manageable environmental-monitoring task using satellite imagery and one complementary data source.
- Compare image-only and multimodal approaches as observations arrive sequentially, assessing environmental monitoring performance, the fusion of knowledge representations and retention of previously learnt knowledge.
- Investigate how prototype representations and cluster preservation behave when the observations or complementary data change and assess whether meaningful relationships between modalities are retained.
