3-D modelling and robot localization from visual and range data in natural scenes

This paper concerns the exploration of a natural environment by a mobile robot equipped with both a video camera and a range sensor (stereo or laser range finder); we focus on the interest of such a multisensory system to deal with the incremental construction of a global model of the environment and with the 3-D localization of the mobile robot. The 3-D segmentation of the range data provides a geometrical scene description: the regions issued from the segmentation step correspond either to the ground or to objects emerging from this ground (e.g. rocks, vegetations). The 3D boundaries of these regions can be projected on the video image, so that each one can be characterized and after-wards identified, by a probabilistic method, to obtain its nature (e.g. soil, rocks… ); the ground region can be over-segmented, adding visual information, such as the texture. During the robot motions, a slow and a fast processes are simultaneously executed; in the modelling process (currently 0.1Hz), a global landmark-based model is incrementally built and the robot situation can be estimated if some discriminant landmarks are selected from the detected objects in the range data; in the tracking process (currently 1Hz), selected landmarks are tracked in the visual data. The tracking results are used to simplify the matching between landmarks in the modelling process.