Multimodal dataset generation through physics-based virtual environment simulation

Advancing autonomous perception for defense applications requires AI models capable of robust object detection across complex and dynamic battlefield environments. Such capability depends on access to diverse, high-quality, high-fidelity datasets that capture the variability of real-world sensing conditions that are often difficult or impractical to acquire through conventional field collection. This paper presents IRIS-EM physics-based virtual simulation framework for generating synthetic, multimodal datasets tailored to battlefield sensing scenarios. The IRIS-EM framework fuses high-fidelity 3D CAD geometries with configurable virtual sensing modules to emulate optical, LiDAR, night vision (NV), and infrared (IR) modalities, where LiDAR incorporates elevation mapping, reflectivity modeling, and physics-informed noise injection to reproduce realistic sensor responses under varied operational and environmental conditions. It enables controlled variation in parameters such as sensor range, viewing perspective, and illumination, ensuring consistent and comparable datasets across sensing modalities. An integrated annotation module supports automated labeling with bounding polygons and adaptive sampling to promote class balance and reduce redundancy. The resulting datasets capture diverse tactical scenes, terrains, and lighting conditions, providing a scalable and reproducible means of training, testing, and validating AI-based object detection systems for defense and surveillance applications.