Artificial intelligence in multi-object recognition and persistent tracking

The authors investigate detection, classification, indexing, and automatic tracking of multiple stationary and moving objects in complex dynamic environments. We conduct fundamental and applied research, as well as undertaking technology developments in machine learning and pattern recognition aimed for intelligence, surveillance, and reconnaissance (ISR). On mini unmanned aerial systems (mUASs), the proof-of-concept onboard actionable intelligencecentric machine vision module is implemented using ultra-compact low-power single-board computer (SBC), cameras, and transceiver. The developed learning algorithms are processed by single-stage end-to-end fully convolutional neural networks (CNNs). The modified variants of You Only Look Once (YOLO) are investigated. The novelty lies in the use of new informative and regularizable loss functions with consequent hyperparameter optimization, as well as devised postprocessing observers aimed to perform robust indexing and dynamic tracking. The data-driven algorithms and factorizations yield trustworthy learning models, improving overall capabilities and enable decision superiority. For quantitative assessment, the ISR-quantifiable metrics are proposed. The metrics are computed to evaluate detection, indexing and persistent tracking for multiple objects. Prerecorded and live videos in complex and congested urban, crosscountry and cross-terrain environments are used to validate and verify findings. High-fidelity balanced datasets are developed. The proof-of-concept ISR-aware machine vision modules are designed, tested and validated in air, land and maritime domains. The modular 8" to 10" quadrotor mUAVs are used as high-tech demonstration platforms to verify functionality, effectiveness and capabilities. In complex environments under design challenges, the object acquisition capabilities are met by integrating onboard machine learning, edge technologies and traditional pillars, such as algorithms and hardware optimization.