December 06 2021

Using an AI-Powered Drone to Autonomously Capture Digital Twins of Tactical Environments

Erik Till

Drones have proven their worth many times over in tactical environments, keeping soldiers out of harm’s way while providing better situational awareness. The value of a small unmanned aircraft system (sUAS) increases by orders of magnitude when it is enhanced with cutting-edge, AI-powered autonomy, opening up even more applications that can empower the operator, decrease cognitive load, support the mission, and even save lives. One example of an advanced use case is the autonomous capture of digital twins in high-risk areas.

Viewing and interacting with this digital twin of a Military Operations in Urban Terrain (MOUT) village, it’s easy to understand how useful this can be to the planning, execution, and evaluation of a mission. This particular model is a good representation of an urban warfare environment, shown with a larger coverage area which lends itself to a wide variety of operational implementations. Military Operations often take place in high-risk or even contested environments, domestically or overseas in combat theater. This detailed map of the scene would be invaluable in the following situations:

  • On-site Key Leader Engagements (KLE): These are an important part of regional engagement, and provide security assessments. They often happen in remote and infrequently visited locations. Any data brought back from these kinds of engagements increases regional awareness and understanding. A digital 3D model allows the insights to be propagated and shared amongst the broader team. These regional trends, help to inform the broader understanding of AO’s (Area of Operations)

  • Training and Communication Aids: The model can be used to document areas around base perimeters, for use in operational training, and as a communication tool. Entry Control Points (ECPs), such as at an airbase or outstation, are often surrounded by urban areas. Overhead satellite imagery doesn’t provide a full representation of a complex environment such as this. This “sandbox model” is as true to the real world as you can get, and is a huge aid in training and communicating what you’re seeing on the job.

  • Post-Mission Documentation: A 3D model of this type, allows for highly detailed debriefs which can help in future operational planning. When discussing what went well, what didn’t, and what can be improved on during a hotwash, this dataset provides the highest level of fidelity possible, resulting in more effective and accurate debriefs. It provides a greater understanding of the scene in after-action reports and enables decision makers to implement mitigation strategies to reduce future risk to the force.

  • Infrastructure and Reconstruction: Giving engineers and specialists the ability to review, measure, and plan based on this highly accurate data that’s been recently captured, is invaluable for supporting reconstruction and infrastructure development efforts. This is a huge leap forward from relying on satellite imagery, which may be outdated or incomplete.

As important as these use cases are, getting accurate data quickly and confidently on the first time can make a large difference in ensuring mission success. Using the Skydio X2D with Skydio 3D Scan software, after a few user-entered parameters are set, the system will autonomously assess the scene and then capture high-fidelity pictures and data, which can be used to build out incredibly detailed and accurate models by processing the dataset into standard photogrammetry software solutions. It’s easy enough for anyone to use with a simple user interface that can be learned in minutes, allowing the operator to focus on the mission.

Set the boundaries and your preferences, and Skydio X2D does the rest from there, autonomously deciding the optimal flight path, executing the mission while avoiding obstacles, and capturing all the necessary imagery to create the digital twin. By specifying parameters such as overlap, offset distance, and Ground Sample Distance (GSD), 3D Scan allows you to get the capture data you need on the first try.

The software will even tell you the number of batteries needed and estimated flight time based on the size of the capture area and desired resolution. With this information, an operator can understand the time and resources needed before starting a mission. This enables better planning, mitigates risk, saves time, and allows users to accomplish more when in the field.

Whether it’s a high-value object, building, scene, or scan of an entire site, no angle is left uncovered with 3D Scan, even with complex geometries. Once the data is captured, it’s as close to ground truth as possible – which is vital for the situations mentioned above – without having to actually be there. You’ll have a complete and detailed record of the scene in high resolution with consistent coverage.

Photogrammetry provides broader macro-level context when zoomed out, and Google Earth or other GIS products can be layered in for even more context. Moreover, the raw photos from the 3D Scan capture have embedded metadata and can be zoomed in individually at the highest fidelity for micro-level inspection.

Autonomous capture of digital twin

Thanks to its onboard intelligence, the Skydio X2D reduces cognitive burden on the pilot, increases the pilot's personal security posture, and augments an overall increase in situational awareness. Designed and assembled in the USA, this powerful system is trusted by the DoD, NDAA compliant, and portable enough to be deployed organically at the unit level. Empowering military personnel with the ability to autonomously capture digital twins in tactical environments, in addition to enhanced ISR, is a force multiplier that will support mission success. If you are ready to bring these capabilities to your team, contact us directly from our website today.