Development and Validation of Feature-Based Vision Algorithms for Autonomous Ship-Deck Landing

Abstract

This dissertation investigates feature-based computer vision algorithms for ship-deck landing. These algorithms incorporate 2D, 3D, and stereo computer vision techniques. The vision algorithms were validated using camera and computer hardware integrated into a quadrotor aircraft. Each of the evaluated vision algorithms is capable of estimating ship-deck pose or ship motion in real-time using only on-board vision hardware. First, a previously developed 2D feature-based algorithm was refined and experimentally characterized. The algorithm utilizes feature matching to estimate a ship-deck pose using a stored 2D reference image. Performance was compared for a realistic helipad reference image and a detailed, customized image. Experimental validation was performed for Sea-state 6 ship-deck motion simulated using a Stewart platform. Additional experiments were performed for visually-degraded conditions, including occlusion, glare, and low illumination, and free flight. The results indicated that while the algorithm can utilize realistic landing pad designs, a feature-rich, custom landing pad image is needed for good performance. With this customized reference image, the algorithm was found to accurately estimate the ship-deck pose across all test scenarios. A 3D feature-based vision algorithm was developed. The algorithm is based on the structure-from-motion technique used in 3D computer vision, and is capable of estimating the pose of a known 3D object. A simplified 3D object was created to represent a ship-deck for experimental validation. The accuracy and computational speed of the algorithm were evaluated for different feature detectors. Experimental tests were carried out for Sea-state 6 motion. The results indicated good pose estimation performance, with comparable accuracy to 2D vision methods. Additional tests validated performance for visually-degraded conditions, specifically glare, occlusion, and low illumination, as well as free hover. The algorithm was found to be robust to occlusion and low illumination, but performance was reduced in severe glare. A feature-based stereo vision algorithm was also developed. The algorithm can estimating the motion of arbitrary ship structures without prior knowledge of their visual appearance or geometry. Experimental results showed that the stereo vision algorithm is capable of accurately estimating the motion of a simulated ship. As the algorithm estimates motion between frames, an initial pose is necessary, and pose estimates will drift over time. A Kalman filter was used to fuse results from the stereo vision algorithm with the previous 2D feature-based algorithm, which provides full estimates of ship pose and does not suffer from drift. This fused algorithm was validated using a realistic simulated ship incorporating 3D structures and 2D flight deck markings representative of a DDG-51 type ship. The fused algorithm was evaluated for Sea-state 6 motion, large-angle motion, and free flight. Results showed very accurate pose estimation results for Sea-state 6 motion, but accuracy was reduced for the large-angle case. In the free-flight test, the fused algorithm relied heavily on the stereo vision algorithm due to failure of the 2D algorithm to detect the flight deck, but still provided reasonable pose estimates.