Automated airborne detection of underwater munitions using NASA multispectral passive and active MiDAR Fluid Lensing

IntroductionWe explored using several novel airborne aquatic remote sensing technologies for the automated airborne detection and localization of underwater military munitions in a complex marine environment. These include National Aeronautics and Space Administration (NASA‘s) passive multispectral Fluid Lensing and active Multispectral Imaging, Detection, and Active Reflectance (MiDAR) technologies, as well as a modified version of the NASA NEMO-Net neural network framework based on a pretrained You Only Look Once (YOLO) model. Unexploded ordnance (UXO) presents an ongoing hazard in international shallow marine environments, with munitions dating to WWI and earlier posing risks not only to humans in local communities but also to marine ecosystems and maritime infrastructure. Remediation of in-water UXO requires localizing and characterizing anthropogenic objects in cluttered environments where the marine environment obfuscates detection because of biofouling, sedimentation, and other changes in UXO appearance and structure. The detection of UXO in littoral zones 10 m and shallower, where they pose the most direct risk to humans, remains especially challenging owing to the limited ability of acoustic methods to operate over large areas in such shallow regimes and the effects of ocean wave distortion and caustics on optical sensing methods from aircraft or spacecraft.MethodsHere, multispectral (444–842 nm) airborne Fluid Lensing, an airborne remote sensing technology capable of optical imaging through ocean wave distortion without refractive or caustic effects, as well as active MiDAR Fluid Lensing, spanning ultraviolet to visible optical bands (375–675 nm), was applied to image underwater munitions of varying colors and conditions ranging in size from 2 to 10 cm in width and maximal linear dimension from 25.5 to 66 cm over a large marine environment using unpiloted aerial vehicles (UAVs). Inert munitions were deployed underwater at the University of Miami’s Florida Keys Broad Key Research Station under a National Oceanic and Atmospheric Administration (NOAA)/Florida Keys National Marine Sanctuary (FKNMS) permit over a large area replete with high anthropogenic and natural clutter. Over the next 2 months, the targets were left to biofoul and accumulate sediment. Airborne Fluid Lensing campaigns were then conducted to detect and localize the targets prior to manually removing them.ResultsWe trained a YOLO-based model on 2,700 artificially augmented samples from nine UXO targets in our field site to detect the inert munitions from the airborne datasets. We detected and localized all 14 deployed UXO targets in three imaging products spanning passive [3-band high resolution (0.5–1-cm Ground Sample Distance (GSD)) and 10-band multispectral (1–3-cm GSD)] and active [8-band MiDAR (0.3–1-cm GSD)] sensing modalities at previously unknown locations. This instance-segmentation detector achieved high precision with moderate recall upon convergence (~200 epochs: P≈0.95, R≈0.71, mAP@0.5≈0.775, mAP@0.5:0.95≈0.488) and cross-validated with an F1 (Dice) score within 0.83–0.89.DiscussionWe found that active 8-band MiDAR Fluid Lensing outperforms passive 3-band and 10-band multispectral Fluid Lensing at comparable spatial resolution, with precision values in the 0.8–0.9, 0.73–0.89, and 0.71–0.74 ranges, respectively. Indeed, several active water-penetrating MiDAR bands were identified for these UXO targets that result in higher precision, even in the presence of decoy targets placed next to the target UXO. Together, these results suggest that airborne active MiDAR and passive Fluid Lensing combined with a pretrained convolutional neural network are viable solutions to large-scale UXO detection in cluttered marine environments; however, additional campaigns and UXO target types are needed to scale the method more broadly and increase detector precision while reducing false-positive rates across more heterogeneity in depth and benthic substrates.