Geospatial-temporal sensemaking of remote sensing activity detections with multimodal large language model

Understanding Earth’s dynamic changes requires automated recognition systems capable of temporal reasoning over remote sensing data. Extending the IARPA SMART heavy-construction dataset, we integrate newly generated text captions to describe evolving geospatial events. By fusing geo-INT image sequences, we train a multimodal large language model (MLLM) to generate descriptive captions and predict future target states. Our unique GPT-based transformer architecture includes a 7-billion-parameter model fine-tuned from a general foundation. We address challenges of sparse satellite collections through data-efficient training and visual question answering (VQA) data augmentation. Building upon prior work in SAR-based ATR and VQA, our results demonstrate that modern MLLMs can generalize across time-dependent sensing domains, advancing geospatial-temporal analysis for improved safety, security, and sustainability.