Visual Navigation System

My thesis project in grad school was the design, implementation, and testing of a system for navigating drones through indoor spaces. The system should address the domain-specific challenges of indoor flight:

• No GPS
• Unreliable Magnetometer and Barometer
• Smaller areas -> tighter control and smaller margins
• Live environment, with dynamic obstacles

I proposed and implemented a system using Visual Odometry for velocity tracking, AprilTags for absolute position control, and software to maintain an occupancy grid of nearby voxels and then automatically route around them using A*. The depth-sensing cameras that generated the VO data also served double duty as a last-ditch collision avoidance system.

A 3D GUI was developed using Three.js, to aid in visualizing the inherently three-dimensional data processed in the drone. Although fun to look at, the top-down 2D map is more useful for command-and-control.

The end result was that you could click on a point on a map, and the drone would get there, adjusting its route along the way to avoid any obstacles it discovered in flight. Computation was done on-board, using a Jetson TX2 and a ZED Mini stereo camera. In the video below, the room has a column in the right third of the screen, which has been deleted from the map. This forces the drone to “discover” the column on its own and navigate around it. Visualization is from actual flight; not simulated.

This final video shows a flight from multiple viewpoints and interfaces. The flight is a bit slow-paced, but demonstrates collision avoidance (trying to drive the drone into a column) and waypoint navigation.

My defense of this research is expected to take place in late August 2020, and my dissertation will be published shortly thereafter.