Investigating crowdsourced neural radiance maps for autonomous vehicles

A key component of autonomous vehicle technology is robust self-localization, which requires machine-readable maps of the road environments. Creating and updating these maps is only possible via crowdsourcing image data from cameras mounted on vehicles driving on public roads. Among the existing 3D spatial reconstruction methods, neural radiance fields (NeRFs) are gaining significant popularity as they are capable of creating an abstract representation of a scene using only a limited number of views and synthesize novel photorealistic views. Once a NeRF representation of a scene is given, one can turn the vehicle localization problem into an inverse problem, estimating the camera pose of a query image in the NeRF. We present a pipeline that enables the generation of accurate NeRF maps via crowdsourcing and analyze different variations in simulation within the CARLA driving simulator. We show that training NeRFs is possible from crowdsourced image feeds of monocular windscreen cameras. We compare 3 different NeRF models, namely TensoRF, Instant-NGP, and Nerfacto, and evaluate the accuracy of the reconstructions based on the number of acquisition vehicles. We also demonstrate interactive visualization of the NeRF model in a virtual reality headset.