We have built a completely new type of camera – one that understands what it sees through artificial intelligence and moves itself with the freedom of flight. It’s a brand new tool for visual storytelling that anyone can pick up and use in minutes — Skydio R1, an autonomous system capable of flying itself and avoiding obstacles all while capturing amazing video of you.
To achieve this ambitious goal, we had to develop custom hardware and software in unison. We engineered a 13 camera system that sees in every direction. The R1 then feeds what it sees into the NVIDIA TX1, a powerful computer with 256 GPU cores that can drive Skydio’s software for understanding and acting in the real world: Intuition AI.
The first step in the Skydio Autonomy Engine vision processing system is understanding where the vehicle is. To do this, the R1 looks at the images coming from its cameras to identify texture in the environment as landmarks, tracking them over time to estimate its own motion with centimeter level precision.
The next step is building a 3D spatial understanding of the environment. With each pair of cameras, the vehicle computes detailed depth maps in every direction around R1. These get fused into a 3D model of the environment.
From there, the R1 uses a deep neural network to identify the subject and other people seen by the forward facing cameras. As it tracks people, it learns what they look like so it can tell the subject apart from other people it sees and stay locked on the right person in the scene.
Once the R1 has built up a detailed understanding of its surroundings, it needs to figure out what to do. It continually predicts as far as 4 seconds into the future and balances avoiding obstacles, keeping the subjects well positioned, and moving smoothly to capture amazing video.
Together, these planning commands get turned into action in the real world as rotor speeds and gimbal angles. It’s a very complex system, but the end result is simple: it flies itself to capture amazing video so you can focus on the important stuff.
We’re incredibly excited to bring the state of the art in robotics to consumers, and our goal is to use this technology platform to make the most exciting use cases for drones come to life over the next few years.