Autonomous Drones for energy and utilities: An EPRI research study
The world’s leading utilities are increasingly turning to drones to assist with inspection of critical infrastructure. From power generation, to transmission, to distribution, the use of drones helps inspectors stay on the ground in safety, only performing risky high-altitude work when absolutely necessary.
- Good morning, uh, good afternoon, good evening,
- depending on where you're joining
- us from.
- Uh, my name is Mauricio Barra. I'm part of the Skydio
- team here. Uh, we're not just getting started
- yet, uh, but for y- those of you that joined us
- early, uh, we wanna get your excitement going by
- showing you, you know, one of our latest videos,
- which can be pretty
- exciting. This is about something called 3D Tower
- Capture. Is, uh, one of our latest features added
- to the Skydio 3D Scan product.
- Uh, and so with that, I'm gonna leave you with a video,
- and we'll see you in a
- couple minutes to get the webinar started.
- [upbeat music]
- Welcome, and thank you for joining us for today's
- webinar, Autonomous Drones for
- Energy Utilities, an EPRI s- research study brought to you by
- Commercial UAV News and sponsored by Skydio.
- I have a few brief housekeeping notes to get us started.
- Um, first, if you have any issues with the webinar
- platform,
- please use the chat box and I will respond right away.
- If you have any audio issues, please make sure your computer speakers
- are
- on, the volume is up, and maybe check that internet connection.
- Otherwise, you can call in by clicking the phone
- icon.
- For those that do need to call in, to ensure call quality, everyone's lines
- have been muted. But we do encourage you to ask questions
- throughout the
- webinar. Please type your questions into the Q&A
- box and select All Panelists when submitting them.
- And at the end of the webinar, we will have our speakers
- answer as many questions as time allows.
- If we're not able to get to all questions today, we'll make sure our sponsor
- follows up via email post-webinar.
- And finally, this webinar will be recorded
- and you'll receive a thank
- you email with the on-demand materials within two business days.
- This will include the live webinar recorded, so you can play back
- as needed, as well as a PDF of the presentation.
- So again, please look for that thank you email in your inbox in the
- next two
- business days. And with that all out of the way, I would like to
- hand the webinar off to the Skydio team. Welcome.
- Uh, thank you. Thank you very much, Nora.
- Thank you for having us, and, and hello again to
- everybody that, uh, is just joining us.
- Uh, my name is Mauricio Barra. I'm the, I'm the senior director
- of, uh, enterprise product marketing at Skydio.
- Very, very excited to, to be here and talk about some
- drones and how it applies to energies and utilities.
- Uh, to give you a little bit of, of, of context into what we're gonna cover,
- um, you know, I'm gonna s- kick off the presentation with just some
- Skydio overview, uh, to make sure that anybody here
- that is joining us that hasn't heard about Skydio before
- or has heard about us before but hasn't learned about the latest
- that we've
- done, uh, we just wanna make sure that you're up to speed, uh,
- with everything related to Skydio.
- So I'm gonna do that in about 15 minutes in the presentation.
- Uh, but then from there, the most interesting part of this,
- we'll be handing it
- off to our experts that we have here today.
- Uh, this is about, uh, talking how these autonomous drones can be
- applied
- to inspection in y- energy utilities.
- So, uh, uh, we're actually gonna have a little
- conversation with two of
- our experts, um,
- Colin Romberger from Skydio and Dexter Lewis from EPRI,
- and so I'm gonna allow them to introduce themselves right now.
- Great. Uh, thanks, Sam. I appreciate it.
- Um, Colin Romberger, uh, solutions engineer at Skydio.
- I've been with Skydio now for, uh, just over a year.
- Uh, and, uh, my role, uh, within the company is, um, really
- to, to provide the technical resources, um, you know, both to our
- customers, uh, as they... as we're going through the process of
- introducing them to, uh, what products, uh, we offer, uh,
- here at Skydio. Um, but then also working with them on the
- customer
- success side, uh, to make sure that once our products
- are deployed in the field,
- uh, we're capturing all the feedback, um, from how they're being utilized.
- Areas where, you know, we still have, uh, um, you know, um, means that we
- could use to optimize our products, uh, to meet specific needs, and
- then working closely with our, uh, product development team to make sure
- that we're
- tracking that and moving in the right direction.
- Um, personally, I've been involved in, uh, unmanned aircraft systems, uh,
- professionally, uh, for, uh, about, uh, seven, eight years at this point.
- Um, got started, uh, before Part 107 back, uh, during the
- Section 333 times. Uh, have a diverse aviation
- background, uh, before getting into, uh, drones as a profession.
- Um, and, uh, also have a master's, uh, in, uh, unmanned
- systems, uh, uh, systems design and also
- operations, uh, from, uh, Embry-Riddle.
- Uh, so very happy to be here today, um, and, uh, to, uh, have a
- conversation with Dexter, um, to share what, uh,
- we've been doing in the utility
- space, uh, over the last almost year, uh, at this point. So looking forward to it.
- Hey, Colin. Hi, everyone. I'm, uh, Dexter Lewis.
- I'm a technical leader for the Electric Power Research Institute.
- Um, in short, I'm a, uh, project manager for collaborative
- electric utility research activities.
- Uh, most of my work area is related to overhead inspections
- and trying to automate that process. Uh, I really enjoy my work.
- I'm always looking for collaborators.
- I, uh, have been involved with drones for about
- seven years, similar to what Colin
- said, and I've been in the electric utility
- research
- industry for, uh, about 15 years. So,
- um, career researcher, which means I don't really know anything
- other than,
- uh, how to experiment and test out some of these new crazy
- ideas. So thank you, and look forward to some conversation with
- you all.
- Thank you, Dexter and Colin. So, um, we'll,
- we'll talk to you in a little bit. Uh, in the meantime, I'm gonna take it off.
- But basically, as you heard, like, the point of this webinar is to show
- you, uh, some of the amazing research that EPRI is
- undertaking with the use of, you know, Skydio autonomous drones.
- Uh,
- so, so we'll get back to that in, in a little bit.
- But in the meantime,
- uh, I wanna, again, start by giving you a little bit of
- context.
- For anybody here that is not familiar with Skydio,
- you know, Skydio is, is a young company. We were founded in 2014.
- Uh, but we've managed to show our worth, uh, by
- becoming, uh, the leading US drone manufacturer by
- volume.
- And really, uh, this is, this is on the shoulders of us really being the
- world
- leader in autonomous flight technology, right?
- So our, our, our flight technology, if you hadn't seen it, it's, it's quite,
- quite
- amazing. Uh, the company became prominent in the
- consumer market, uh, thanks to our, uh, our drones that can
- be pretty amazing for people that are into action sports
- and want a drone that can autonomously track them, uh,
- while
- avoiding incredible obstacles.
- Uh, but in, in 2020, we actually expanded our scope,
- uh, because we realized that the, the value of these autonomous flight
- capabilities was, was even, even bigger when, when you
- thought of applying it, uh, to professional use cases, right?
- Where, uh, where the people, you know, trying to get a job done, they're
- not really trying to, to fly drones, right?
- Like, they're just trying to use drones as a
- means to an end.
- And so, and so these, this, this, these companies that are trying to use
- drones to get this job done see the value in autonomous flight.
- And so we have some pretty amazing customers already.
- We have already gathered over 600 enterprise and public
- sector customers, uh, at this point who believe in this vision
- for, for autonomous flight, and so they're embarking with us on this
- vision.
- Uh, and we've also been thoroughly recognized
- by the industry
- as being, uh, an innovator in this space, right?
- And so Frost & Sullivan, uh, recognized us as the Drone Company
- of the Year in 2021. Fast Company also,
- like, recognized us as one of the most
- innovative companies in 2021.
- Uh, so as you can see, uh, we- we're here to stay.
- Uh, we're here to, to, to make a big splash,
- and, and, and we're doing so. But...
- So Skydio was really, you know, founded with the idea that, you know,
- drones have
- this amazing potential to do good in the world, right?
- To, to, you know, to make the world safer by allowing,
- you know,
- professional operators to n- not put themselves in, in harm's way,
- and instead just put a drone up in the air and try to capture the data that
- they need. Uh, we believe that we can make the world more efficient,
- uh, more creative.
- Uh, and, and that's the potential for drones.
- But the reality is that,
- um, you know, for, for us to be able to achieve this vision,
- uh, we really need to rely on, on the power of automation and
- autonomy, right? Because if you think about the way drones
- operate today, uh, the reality is they're still pretty,
- pretty complex, pretty costly and risky to operate.
- And so while they are amazingly valuable tools,
- uh, the fact that, you know, drone teams still need to
- put all this work into learning how to operate this, uh, these tools
- safely, they still need big teams to do so.
- Means that it, it's just, it's just a little bit difficult to scale,
- right?
- And to make this the standard way of doing
- the most complex inspections that industry has to face right now,
- right?
- So, uh, when you think about it, really, like drone use
- is still a little bit niche, right?
- Like, there's still a lot of things being done
- with heavy machinery or with
- people doing incredible climbs. And so, uh,
- and so for us to be able to get to a point where
- drones are the de facto, uh, method
- for, uh, capturing data, uh, that requires
- aerial vantage points, uh, we need to get away, right, with
- these complexities, uh, that, that, that ultimately means
- that you're putting more, more hours into learning and
- operating the tool than the beneficial hours that you're getting out of the
- tool itself, right? And so until we get there,
- we're not gonna be able to drive adoption.
- So, so Skydio, um, uh,
- approaches the problem in a fundamentally different way, right?
- Like the way we started was by building the
- software underneath, right? Like the, the drone around it came
- second, but first and foremost, we wanted to
- build something that could, you know, uh, navigate itself
- autonomously so that it can support the pilot in
- command at all times. And so this is the concept of AI-driven autonomy,
- right? The fact that we can allow the drone,
- uh, to, to assist the pilot at all times by
- allowing the drone to cover these three core tenets or
- concepts of autonomy, right? The first one is we need to enable the
- drone to see in the exact same way that we
- see when we're trying to get something done, right?
- So to be able to do that, our drones are equipped with
- cutting-edge, uh, cutti- computer vision algorithms,
- uh, that are being fed by images captured by
- six navigation cameras mounted around the drone.
- And so the drone is capable of seeing in every direction, is
- capable of sensing everything that's happening around it.
- From there, though, the most important piece
- is then the drone needs to be able to
- make sense out of that information, be able to understand it.
- And so for that, our drones are equipped with, uh, deep
- neural networks, uh, that are ultimately building
- a 360-degree map of the drone's surroundings
- at all time. It's cap- the drone is capable of recognizing subjects,
- is capable of inferring shapes and predicting into the
- future so that ultimately the drone is able to
- take intelligent action to support the pilot in
- command, right? And so at all times, our drones are capable of
- avoiding obstacles, again, in 360 degrees.
- And also, they are capable of automating entire
- actions, right? So like tracking a subject or
- essentially automating the capture of an unknown structure,
- just, just plotting a path around the structure
- and capturing the
- images, like the way we do with our Skydio 3D Scan
- software,
- right? So,
- so that is our vision for what sky autonomy is,
- and... But obviously, I can only tell you so much.
- So therefore, I wanna show you just a video so that you
- can understand, uh, what it means
- to, uh, to implement autonomy in these concepts
- that I just
- showed you all put together.
- So let's take a look.
- So the first view that you're seeing here is the, uh, the
- navigation camera, right? That, and so on the left is what
- the drone sees.
- Then from there, the drone is capable of creating this voxel map,
- so that it can understand the drone, uh, the, the world around it.
- And as I talked about also, is capable of actually predicting
- into the future to implement these incredible tracking capabilities.
- And because of the computer vision system, this, the drone is also capable of
- flying where other drones simply can't, right?
- Because it's not relying on a compass, on a
- magnetometer, on GPS connectivity.
- We have all that, but we don't rely on it for navigation
- because
- the drone is navigating with computer vision.
- So as you can see, it's pretty amazing
- stuff. And ultimately, what it translates
- into is that our drones
- are, are, you know, different from any other drone in the
- market,
- uh, around some of these concepts here
- that I wanna explain, right?
- So our drone is capable of getting up in the air faster.
- Because of that computer vision system, our drones don't need to
- rely on that complicated calibration dance that many other drones
- need to get the compass going. We don't need to do that,
- so you can just hand launch the drone in a matter of seconds.
- Because of the obstacle avoidance, our drones are inherently
- easier to fly and safer from crashes.
- That's just the way it goes. And again, because of that computer vision system,
- our drones are gonna be capable of capturing better data, right?
- Like for example, the ability of flying these, in these
- environments that are, are heavy with metal, that have
- radio,
- radio frequency interference, electroma- electromagnetic interference.
- Our drones are not gonna be affected by that, are gonna be able to fly
- steadily through that. And we can also get much
- closer, uh, to structures so that you can capture, again,
- the details that you
- need, again, because of that amazing obstacle avoidance.
- And ultimately, I apologize for this slide.
- Like, we usually have some GIFs here that are not playing in this
- particular, uh, way of showing the presentation.
- But ultimately, the last concept is about automating entire
- workflows like the way we do with Skydio 3D Scan, so that you
- can essentially, you know, increase your efficiency of capture.
- So,
- uh, so as you can understand already, you know, as I mentioned before,
- you know, we do things differently.
- We start from the software and work ourselves up to everything else.
- So we are a software company at heart.
- And so while we do have amazing drones as for the foundation for
- everything, with the Skydio 2, the Skydio X2,
- um, ultimately our goal is to equip them with fantastic
- software that can help you get things done in the most
- efficient way possible. Uh, and ultimately, to all of
- that, you know, have it connected to the cloud so that
- you can ultimately get the data that you're capturing
- as fast as possible, uh, to the teams that need to get access to it,
- right? So
- that is why we've also have the whole, I don't know, Skydio Cloud
- service up and running with fleet management capabilities, media
- sync, and a few other capabilities in the making
- to, again,
- allow you to take that data as fast as possible to your most
- important stakeholders.
- So as we dive a little deeper into our solutions, at the high level,
- these are the drones that we have in the market today.
- Uh, the Skydio 2 Plus, uh, is, is, you know,
- the, the drone that came after the Skydio 2,
- which is the drone that really
- allow- allowed us to establish ourselves, is the drone that really change-- like
- set the standard for autonomous navigation in the consumer market.
- Uh, but now, uh, you know, it's being used by all
- kinds of enterprise and public sector agencies that realize
- the value of, of autonomous flight, and this is a, an incredible drone,
- you know, at a really attractive price point.
- And so very easy to get started with a drone program leveraging these
- capabilities with the Skydio 2. And the Skydio 2 Plus is a drone
- that we announced
- in January. In a second, I'll cover it in a little bit more detail.
- And then we have the Skydio X2, which is really the, the, the
- bigger brother to the Skydio 2, is our, is our beast,
- as we used to ca- uh, we like to call it.
- It's really an enterprise workhorse that is designed to really
- tackle successfully the most demanding, challenging environments that
- you can face.
- So going a little bit deeper into these drones,
- the Skydio 2 Plus, as I said, is a drone that we introduced in January
- of this year. And again, it's still powered by Skydio Autonomy
- Enterprise, our, our, our autonomous flight engine.
- Uh, it still has all the amazing capabilities
- that you know from the
- Skydio 2, like three hundred and sixty degree obstacle avoidance,
- the ability to fly in GPS-denied environments, the ability to
- track subjects autonomously. But it comes with some
- amazing enhancements. For example, Skydio 2 Plus
- now has longer battery life with up to twenty-seven
- minutes. And actually, that, that battery life is not exclusive to the
- Skydio 2 Plus because that battery is backwards compatible with the Skydio
- 2.
- Uh, but specific to the Skydio 2 Plus is that it also has a better radio, and
- it comes with those antennas that you see there so that y-
- it has much better
- wireless range that now can extend up to six kilometers,
- uh, which is way up from what it used to be
- before.
- And finally, for professional operators,
- it's also the fact that now the Skydio 2 Plus is compatible
- with our Skydio Enterprise controller, which if you've seen our webinars before,
- it's a controller that we use to position for our Skydio X2
- platform. But that, that ground control system can now
- also be used for the Skydio 2 Plus. So overall,
- really amazing updates, uh, to, to the Skydio 2 Plus
- platform.
- And, and so now, uh, let me move on to Skydio
- X2. So the Skydio X2, as,
- as I said, is our enterprise drone, also powered by Skydio Autonomy
- Enterprise software. Uh, but this one is, uh, is
- built with a different mindset to, to make sure that it can tackle any, any
- sort of challenge. So from a build standpoint,
- it's a drone that is, you
- know, built with, you know, durability at heart, so it's
- ruggedized with carbon fiber, carbon fiber materials.
- It's also, um, designed with foldable
- arms so that it can pack up really small and can deploy really fast.
- It's, uh, it's ready to operate day or night,
- which means that it's capable of GPS-based night light.
- It's also, uh, has strobing lights to be able to,
- you know, de-conflict at, in the nighttime.
- And, and ultimately, it also comes in a couple of variants
- when it comes to the sensor payload.
- So, oh, by the way, it's also NDAA compliant, right?
- Which means, you know, it's, it's fully, you know, secure from a
- supply chain standpoint. And for anybody that, you know, is
- using, you know, here, federal grant money or whatever, it
- means that you're good to go in terms of using this platform because it's NDAA
- compliant. Uh, and so as I mentioned, the other thing about the Skydio
- X2 is that it, it's available in a couple of variants
- when it comes to the, to the
- payload for sensors. So we have it in color thermal.
- So it has both a, you know, a color
- camera that has a more standard, uh, you know,
- l- lens when it comes to the, to the horizontal field of view,
- about forty degrees. Uh, so really optimized for
- situational awareness, like long-distance capture.
- And it also comes with a thermal camera, FLIR
- 320 by 256. Uh, both of them
- have digital zooming capabilities.
- The, the color camera has 16x digital zoom.
- The thermal has 8, 8x, uh, digital zooming
- capability. But then we also have the color,
- the X2 color,
- which removes the thermal camera and adds a color
- camera that is really tailored for more up-close
- capture with a wider, uh, lens, wide-angle
- lens. Um, so that means that this one is more of an
- inspection drone. Which is really ideal for capturing
- with Skydio 3D Scan, uh, which I'm gonna talk about in a
- second if you haven't heard of before.
- So, uh, we have these couple of variants in the X2
- platform
- to really cover the needs of, of, of your inspection.
- A reminder that our drones are, you know, all designed,
- uh, assembled, and, uh, and
- manufactured in, in the United States.
- So that's really good, uh, in terms of making sure that you have
- all the assurances from a supply chain security standpoint.
- And specifically, the X2 is fully NDAA compliant,
- which means that, for example, it has been, you know,
- vetted by the federal
- government as one of the bluest UAS, uh, platforms.
- It was recently selected by the US Army
- as the drone that they are gonna go with for their sh-
- short range
- reconnaissance program. So out of, uh, 30 companies that bid for that,
- the X2 was the drone that they chose, uh, to equip their
- soldiers. So just a testament to both our
- ability to produce, our ability to produce securely, and our
- ability to produce a performant drone.
- Um,
- and, and finally, uh, Skydio 3D Scan, right?
- Like, this is our drone.
- Our... Uh, um, I apologize again, like, we usually have a, a little bit of a
- visual here, which is not showing.
- But Skydio 3D Scan is our adaptive scanning software solution
- to automate the capture around complex structures, right?
- So this software, which can be layered on top of both the
- Skydio 2 Plus
- and the Skydio X2, will allow the drone to fully
- automate the, the process of
- documenting any structure, capturing images of any
- structure in a fully auto- automatic way, right?
- So what it does is that it allows to scan a structure without
- any
- prior knowledge about it. The drone will autonomously
- scan the structure, understand its geometry, its shape, and
- from there,
- plan a flight plan around it to make sure that it
- can capture all the images needed to, so that you can see every angle of that
- structure. So again, complete coverage.
- It will,
- it will do it in sub-millimeter resolution if that's what you want.
- So you define the ground sample distance, and the drone will
- ensure that very consistent ground sample distance as it captures
- it. And so this is a fantastic tool for
- automating
- capture and ultimately automating inspecture of, inspection of any
- structure. And ultimately can also lead to better
- photogrammetry, uh, reconstruction if that's what you need to build a 3D
- model, because we can feed all that data set of 3D
- scan into any standard photogrammetry software like, you know, what you
- see in this slide. So ultimately,
- what you get is this, right? Like, it's robotic precision with consistent
- data. What you see on the images on the left
- is somebody trying to
- capture a pole based on manual flight.
- And obviously, you know, a human pilot can only do so much, right?
- Like, it's, it's difficult to really capture
- every angle of a structure in the, again, in the
- resolution that you need manually.
- It's just, just very complicated and if you, you're gonna sacrifice
- accuracy for efficiency of capture.
- So the good thing with Skydio 3D Scan is that because it's based
- on ar- artificial intelligence and automation, it's going to do it in
- this very, very predictable flight pattern
- that you see on the
- right, and it's gonna do it in, in a matter of minutes, right?
- So you're getting both the, the fast capture as well as the
- efficiency, the accuracy, all of it thanks to the robotic precision
- that our capabilities are capable of delivering.
- And so with that, this is my overview about the capabilities.
- I'm gonna pass it on to,
- uh, you know, to Dexter and Colin to talk
- about how EPRI is using our drones, uh, for
- en- energy utilities. Dexter?
- Hi. Thank you, Mau. Uh, hello everyone.
- And, uh, the title of my slide pretty much says it.
- My work area is related to automating electric utility,
- uh, overhead inspections.
- Uh, I'm gonna hope you find this presentation
- interesting, and, uh, we'll
- just get right to it. So on the next slide,
- you'll see an outline
- for, uh, what I'm gonna discuss today.
- Uh, I'll start by telling you a little bit
- about EPRI, then I'll discuss utility
- inspection and the, uh, opportunity for automation.
- I'll show you a few videos from some tests that we've done with Skydio,
- uh, and then I'll point you to some ongoing
- work and some upcoming work.
- Uh, please add any questions that you have to the chat,
- and I look forward to
- engaging with you, uh, during that Q&A session.
- So on the next slide, you'll see a, a quick overview of the Electric Power
- Research Institute. We were founded in 1972.
- 2022 is our 50th anniversary. We are a independent
- nonprofit energy research and development organization.
- Uh, we collaborate with over 450 companies in
- 45 different countries, and our main goal is to perform
- research and, uh, drive innovation to ensure that the
- public has access to clean, safe, reliable,
- and affordable electricity across the globe.
- We're a trusted resource, uh, because we work objectively.
- Uh, we report facts, share data, and follow the
- scientific method, uh, at least as best we can.
- Uh, through our research, some of which you'll hear today, we identify
- issues, technology gaps, and broader needs in the
- utility industry, and then we do our best to try to address
- them for societal benefit.
- Uh, we don't work alone. We collaborate with utilities and
- academia, private industry, Skydio for example, and the
- general public. Our tagline is, uh, "Together shaping the
- future of energy," and I'm a believer in that power of
- collaboration So next slide I'll, I'll just touch on, um,
- uh, as a nonprofit and an independent organization, our
- primary purpose is to conduct R&D, again, for the benefit of the
- public. Uh, the work I'll describe today is focused
- on drone-based inspections of utility assets, and the
- goal is to advance that utility inspection process.
- In doing so, the public, uh, may benefit by
- reduced outage times, uh, improved safety of the
- workers performing this work, and then, uh, identifying
- issues before they happen, uh, improving reliability.
- I'll note that EPRI does not design or manufacture drones
- to support this research. Uh, instead, I mostly test
- and evaluate prototype or, uh, commercially available
- solutions, but in a somewhat, uh, unique high
- voltage, uh, utility environment.
- Uh, I will call your attention, and it's important to recognize that
- while we do
- work and collaborate with vendors, contractors, and consultants,
- uh, we don't endorse nor disparage the use of,
- uh,
- these entities.
- Uh, our overarching research goal, regardless of the project,
- uh, is to be accurate, objective, and factual.
- Next slide.
- So throughout my presentation, I may say transmission or
- distribution, and when I refer to that, I mean the overhead
- wires and structures and components that support the
- delivery of energy from one place to another, uh,
- typically over long distances. The National
- Academy of Engineering estimates that the, uh, electrical
- delivery system is over two hundred thousand miles for transmission and
- five point five million for distribution.
- They consider it the greatest engineering achievement of the twentieth century.
- Um, now, the main difference between transmission
- and
- distribution is the voltage primarily, but then also the
- amount of energy transfer capabilities.
- Uh, in short, transmission is higher voltage, more
- energy, larger structures, and for lack of a better term, just more
- important. Uh, a decent analogy to this is kinda like
- arteries and veins. So you could think of transmission as your
- arteries and distribution as your veins.
- Uh, and just as many of us take for granted
- that our heart is
- going to pump blood where it's needed, uh, many take for
- granted what it takes to reliably deliver energy
- over these large distances. Um, I at least have
- come to expect when I plug in my phone that the, uh, that it's gonna
- charge.
- Um, so to kick off my presentation, I think I'd agree that it
- is quite an achievement to design and build this
- infrastructure, but I think it's an even greater
- achievement to
- reliably maintain it. Uh, on the next slide, you'll see
- why. And really, it's, it's that these assets, uh, are in
- some of the harshest environments in the world.
- And additionally, there are other variables that really make
- this, uh, maintenance task interesting.
- For example, I've got several pictures here.
- Uh, woodpeckers bore holes in poles, uh,
- degrading the mechanical integrity of that asset.
- Birds nest in structures that contaminate insulators and cause
- flashovers, and bored hunters shoot insulators and
- conductors, and heavy machinery breaks guy wires.
- I mean, long story short, utility assets
- degrade over time, but utilities know this
- and are proactive in their maintenance decisions to stop
- most outages from occurring.
- Uh, but to know what to fix, they've got to, uh,
- perform a visual inspection, and that's really why they do these
- routine inspections.
- Now, the inspection tools and methods do vary, uh, but
- most of the time it's a non-contact visual
- inspection of the structure and its components.
- Um, and they use that information to assess the health of
- individual
- components and basically support a maintenance investment or
- decision.
- Now, drones are great tools to get these
- pictures, but because of the amount of
- infrastructure, remember the five hundred, excuse me, the
- five million miles plus of assets out there,
- uh, inspec-inspections are, uh,
- expensive, and they're time-consuming, and they're
- hazardous, they're hazardous when you perform work in the field.
- So that's really where our research comes in and why we're looking at
- new inspection approaches.
- So the next slide starts by kinda describing our research
- goal, and really it's to optimize the inspection
- workflow. Our hypothesis is that automation
- provides the greatest opportunity for safe,
- fast, low cost, and high quality inspections.
- Uh, automation with drones to capture the
- right imagery safely,
- consistently, and quickly. And automation with
- machine vision or AI to review the imagery or
- video collected and identify issues.
- And note that an inspection is both the combination of
- data collection and data interpretation.
- Um, now, we're using drones because they're
- well-suited for
- overhead inspections. Uh, they're flying sensors,
- usually not super expensive or large and fairly easy to
- operate. Uh, we're using machine vision or AI or ML
- because it may be a solution to the big data problem, you
- know, collect all this information on these
- assets.
- And we're trying to automate both workflows because one day it
- may be better than humans doing it, uh.
- Now, the next slide will be brief, but I wanna describe some of the work
- that we've
- done with Skydio to investigate this automated data collection.
- And let me start by saying that utilities can be hesitant to
- try, uh, unproven approaches in a
- critical infrastructure environment.
- Now, I didn't say this before, but EPRI was born out of the
- 1965 great northeastern blackout, which left, uh,
- 30 million people without electricity.
- And blackouts are bad, and technology demonstrations
- that result in blackouts are bad. So that
- being said, it, it,
- it would be a shame if, um, EPRI had to close our doors because of
- a research project that caused a blackout.
- So while flying in a real operating environment is the end goal,
- we start in our high voltage laboratory and then move
- into low risk environments before we eventually try
- something in an energized operational environment.
- Now in our, our next slide, I'll discuss some of our lab testing,
- and we
- will have a video to play. I'll cue you on this, Mal,
- and I'll also warn the
- audience that, uh, there is some, um, white
- noise in the background. You may wanna be ready with your volume.
- But, uh, just give me a minute and I'll describe the, the test.
- So, uh, electric utility environments are usually
- accompanied by high voltage electric fields and strong
- magnetic, uh, strong magnetic fields induced by current
- flow.
- Uh, to reliably operate autonomously for
- utility
- inspection, a drone and its onboard sensors
- must be able to withstand those fields.
- Um, this first video that I'll show you, it's less than 20 seconds, and it
- does, again, kinda contain that audio, so be ready with that
- volume. Um, this is the sound of one of those
- nasty high voltage electric fields.
- The long bar that you'll see in the video is, uh, energized
- at 230,000 volts, and the drone, in this case
- one of the Skydio X2s, is hovering right above it.
- The white noise that you'll hear... I'll let this video play.
- [white noise]
- This is run two on the X2E.
- Okay. I hope y'all, uh, I hope that didn't deafen anybody there.
- Um...
- [laughs] Dexter, uh, real quick, uh, just, uh, to make sure everyone knows,
- that, uh, vertical movement at the end there,
- that was, uh, pilot commanded, right?
- That was not, uh, a result of, uh, the energy, uh, that we were
- in. Um, you know, a- again, one of the benefits of our systems
- is that, you know,
- we do not utilize those magnetometers, uh, as your slide mentions, uh, which
- is great. Um, and I'm just curious.
- Um, you, you may get to this in your slides, uh,
- as well, but, uh, you know,
- looking at some of the defects, uh, that we saw earlier, um, you know, in
- your, in your experience, you know, how beneficial is it to be able
- to fly closer without the risks, uh, and get, you know, those
- proper angles as opposed to just rely on higher megapixel
- cameras and, and staying further away?
- Do you see a benefit, you know, in being able to do that?
- Uh, yeah. Um, at the end of the day for both
- transmission and distribution structures, there are
- connection points, um, that are common
- failure points, and you have to be able to see, uh, the
- integrity of those connections from multiple angles and from
- unique perspectives. Some of them are hidden underneath cross arms
- or, uh, you know, tucked away in some tight spots.
- So, uh, being able to deploy a drone autonomously
- that can with- handle close operations and not be impacted from
- electromagnetic environment is a requirement to
- deploy,
- to, to use autonomy, uh, for inspections.
- Great. Thank you.
- Okay. Um, so the, the white noise you heard in that
- video was from, uh, an electrical phenomenon called
- corona and an ambient arcing source.
- And we purposely created those, uh, low and
- high frequency fields to try to endoy-
- induce noise into the aircraft's command and control.
- And to Colin's point, um,
- we did not see any negative operation on the aircraft, um,
- in both routine operations as well as, uh, in some of our
- emergency return to home procedures. Everything worked how it was supposed to.
- Now, the, uh, second video that you can cue up,
- and I can talk through this one
- 'cause there's no, uh, no audio here.
- Uh, this is a, a video of our, uh, mock substation environment.
- Video's, uh, a little less than 60 seconds, and it
- shows the Skydio aircraft operating around a
- three-phase 138,000, uh, volt,
- uh, source and a 500 amp current loop.
- So you, we, you gotta have the electric field
- and the magnetic field to
- accurately represent the, the environments.
- There's also quite a lot of obstacles here.
- Uh, this was a 3D scan of the entire yard.
- Uh, we started this work de-energized to
- ensure that the air-
- aircraft could navigate through the environment without any collision, and
- then we energized the environment and repeated the work.
- And I'll again say, in all our energized lab testing, we
- did not observe any electric or magnetic field impact
- to the aircraft's flight behavior, uh, collision avoidance
- systems, or the command, uh, command and control
- systems.
- Um,
- so again, we start in the lab to make sure there's
- safe operation, then
- we go into a, a,
- a, a next environment. So on the next slide, uh,
- this really builds upon our work where we moved into what I'd call
- the, uh, wires environment. And you can cue this video, and
- I'll, I'll talk through it. But, uh...
- Now, the, the wires environment is energized and it's good to have
- aircraft that can handle that electromagnetic environment,
- but it's also an
- obstacle, uh, ridden, noisy environment.
- Um,
- for automation to be successful in the utility environment,
- drones must be able to avoid, uh, wires,
- poles,
- vegetation, and, you know, really any other obstacles.
- Uh, now this was a video from a de-energized
- demonstration at
- a utility training environment. I can see the resolution's a little
- low, but it's, it's one drone up above watching an S2
- and then the first person view of the S2 down in the bottom
- right synced up together.
- Now these stru-
- Yeah. Uh, if folks have the ability to go full
- screen, that, that makes it a little
- easier to see. But, uh, anyway, yeah.
- Thanks, Colin.
- So these structures in this training environment are built to spec,
- and it really just provides a safe playground to experiment with some
- of these automation workflows. Uh, over the course of three
- days, we performed, uh, 60 of these 3D scans on
- different structure types and experimented with different scan
- settings. Um,
- some of these settings and variables that you can manipulate are your standoff
- distance, which is related to your, your spatial resolution,
- um, the rotation axes, the payload pitch,
- overlap settings, and we tried single structure scans
- versus multiple stru- structure scans.
- Um, and if you were kind of watching that video, you, you may have seen some
- overhanging tree limbs that provided a, a pretty interesting,
- um, collision avoidance test.
- So, uh, again, kind of building upon this, after the
- successful demonstration in the, the energized lab environment
- and then the successful demonstration in the utility training
- environment, the next slide shows, uh, some of the
- work that we did when we moved into a operational
- environment.
- So this is a, uh, electric distribution capacitor bank,
- uh, that's feeding a real-time customer.
- So you can go ahead and play this video.
- What you'll see here is the drone flying autonomously around
- this capacitor bank, again, dodging wires, uh,
- capturing imagery. Uh, this is a, a, about a
- one and a half minute video played at regular speed, and if you
- see a small flash during the video, that's the camera shutter
- capturing image stills.
- Um, with this,
- uh, adaptive capture workflow, what we did is we defined a
- operating volume, uh, and allowed the drone to build a
- basic understanding of the, uh, 3D shape of this
- structure. And with that shape, again, you can go in there and
- define those image capture parameters, uh, as some of those I've
- mentioned. And for this demonstration, this is a
- two-axis rotation at a 10-foot standoff with a
- 50% overlap, um, which in turn
- resulted in about a minute and a half flight,
- um, and 48 images or about half a gig of data.
- Um-
- One thing I do want to add in here, uh, Dexter, you know, th- this was,
- um, you know, a, a real roadway, right? An active roadway.
- Um, just, you know, at the location we were flying, uh,
- you know, along the side of
- the road. You can see the vehicles [laughs] passing
- right there.
- And one of the benefits of, you know, our, our 3D scan-based
- workflows, uh, 'cause we do have multiple, you know, that we're optimizing for
- different use cases, utilities being one of 'em, is
- that in addition to
- defining, you know, the area, the volume that we want the, the
- drone to capture, we can also, um, say, "Hey, don't
- go outside of these boundaries." Uh, so one of the things we
- were doing there for
- safety was setting a floor, uh, of that volume high
- enough that the aircraft, you know, would, uh, you know, make sure that it
- remained in a position where any passing traffic, uh, and things, you know, would
- not be, uh, at risk. So, you know, looking at, um, waiver
- requests, et cetera, uh, you know, related to flights near these roadways
- with, uh, distribution, um, you know, I, I feel that, uh, you know, that is a
- benefit, um, you know, in, in this case of, uh, the,
- the current capabilities of
- the system.
- Thank you, Colin.
- So, um, on the next few slides what we'll do
- is we'll take
- a look at some of the imagery that we captured.
- Um,
- of those 48 images, some of them are better than others, and
- reviewing the imagery, what we can begin to do is build an
- image capture, uh, specification.
- Uh, these really define how, where, and what to
- look for. And in all honesty, these are useful for humans, but
- the goal is to translate this information to machines.
- Uh, so a view at this height is certainly valuable.
- Uh, here you can inspect the sensors, the arresters, the pole
- top,
- conductors, arms, brackets, all, all kinds of stuff.
- Uh, and I'll give you bonus points for anyone
- that sees the missing wildlife guard
- on one of the arrester. Uh, if you don't see it, it'll be on the next
- slide we can, we can go to. It's a little bit easier to see.
- Um,
- it's the, it's the arrester at the top left.
- So, you know, an image at the top of the pole provides
- value, and then another
- image that's maybe directly below the conductors looking at the assets
- installed there provides value. Uh, the next slide.
- Um, and really while you're out there, it's not just what's
- on top of the pole, it's really the entire pole
- and also where it makes connection
- into the ground is valuable for an inspection.
- You wanna look and see if you've got any deterioration down at
- your, at your ground line.
- So, uh, the next slide, really what we came out to is of those
- 48 photos, um, we kinda optimized it to
- 12 is what we needed to perform an inspection.
- And really what that translated to was a look at three different pole
- heights, um, and then four different quadrants
- to, again, get a good view of the structure.
- Now, I had to manually look through these
- images and pick
- out these optimal 12, and really where we're trying to take the
- research is have machines or software that can do that.
- If you had that capability, you could embed it on the front end to
- try to optimally only capture these 12 images, which
- has benefits in reducing your time on site.
- Uh, or as well as using it on the back end to
- reduce the amount of
- images that a human would have to look through to perform that inspection.
- So, um, I'm gonna jump now into some of
- that machine learning,
- uh, topic, uh, and I'll just pause here and, Colin,
- if you've got any other,
- uh, thoughts before I go into that.
- No, I, I think you can, uh, keep rolling, Dexter.
- You know, the only thing I'll say is that, um, you know, we've, we've, uh, taken
- heed, uh, of the feedback, you know, both from, uh, you and, and our work with
- EPRI as well as, um, you know, our customers, uh, that are currently,
- you know,
- using, uh, the, the systems, you know, other utilities, uh, that, uh, um,
- are utilizing Skydio products in the field right now.
- And that is, you know, universal. Um, you know, feedback that we've received
- is that, you know, the image... the number of images, you know, for each
- structure, uh, you know, we need to have the ability to optimize
- and, and
- select, you know, which images are being captured.
- And, you know, again, that is the great thing about, you know,
- being a, a
- software-focused, uh, product, um, and having the
- level of visual intelligence that we have baked in.
- Uh, you know, we do have the ability to continue to
- optimize our capture
- modes, which is what the team is doing right now, so that, you know,
- users can,
- um, you know, in the future, uh, select, you know, what areas, what
- quadrants, uh, what information is gonna be most relevant, um,
- and, you know, customize their capture profiles, uh,
- to achieve that.
- So I'm looking forward to seeing what the team is coming out with.
- Okay.
- So on the next few slides, I'll, I'll tell you what EPRI's doing to try to
- help Skydio and really just the entire industry in this space.
- Um, so the hypothesis is that,
- again, machine vision can optimize the data
- capture on the front end
- as well as the post review. And if you look at these pictures, you can
- probably leverage some existing knowledge of
- distribution, um, and you may be able to imagine
- where that camera was when it captured these images, you know,
- in
- reference to that pole.
- Uh, again, and additionally, you can probably look in the images
- and pick out some
- items of interest, like the transformers and, and fed there.
- So the... again, the hypothesis is that computers, we
- can train computers to do the same thing, um, look at these images
- and
- understand their orientation. And I'd say this is an area where EPRI's,
- you know,
- really helping and leading the industry.
- Um,
- there are, uh, really several AI a- AI
- datasets out there, but very few of them
- are actually related to
- overhead transmission and distribution imagery.
- So,
- uh, we have a, a, an unlabeled distribution image
- dataset that we've made free and publicly available.
- It's, uh, 30,000 overhead images if anybody's wanting to experiment and
- research in this space. We're trying to label that
- and according to the
- schema that you see down below, and then publish that publicly as well,
- probably a couple months from now.
- If you're interested in any of that stuff, just, uh, my email information's towards
- the end of the presentation, just, just shoot me a note.
- So the next slide, uh, from a few slides back, I, I
- told you that, um, imagery from four different quadrants at
- three different heights had value.
- And using the conductors, the poles, and the cross arms,
- and a defined reference system, we can, uh,
- identify the viewing quadrant of different images.
- Uh, get a good look at these four quadrants
- and the three heights.
- You don't have to commit them to memory, but I plan to use this example
- for this next slide here.
- So, um, uh, on the next slide there,
- Mel. Thank you. For, uh, an optimized
- adaptive capture flight plan, uh, we may be able, again, to
- leverage that machine learning object detection models.
- Uh,
- it's, it's...
- to do this, it's really important to accurately
- identify
- the pole, the cross arm, and the wires at a minimum.
- If a machine vision model could do that, then it could begin to look for
- relationships that are unique to a certain perspective
- or viewing height. Um, I put a few of them in here that an
- AI model could leverage.
- Uh, you may see some words like obscuration
- and trun- truncation.
- Those are attributes of the individual objects and their classes.
- Uh, they indicate if something's blocking an
- object or if the
- class or object extends past the image field of view, that would be
- truncation. So for example, in the three rows, a, a
- model could determine the viewing height just looking at the pole class if
- it's truncated at the top or at the bottom of the image.
- And also the viewing quadrants could be determined by the conductor
- direction and identifying if the pole is obscuring the cross
- arm or if the cross arm is obscuring the pole.
- Now, these are examples of a best case simple
- pole, uh, scenario, but the distribution
- system and transmission system, uh, is complex.
- And [laughs] on the next slide, you'll kind of see that.
- Uh, in order for this to work robustly and repeatedly,
- uh, models need many examples labeled
- consistently to deal with these complexities.
- And I get asked frequently how many images are needed, and it, it really does
- kind of depend on the specific task and what level of performance is
- acceptable. But at the end of the day, I'll just be honest and tell you, I don't
- know. Uh, it's kind of like asking how many hours
- does it take to learn
- how to play the guitar. I mean, it just depends.
- Are you playing four chord country songs for your mother or
- performing on stage with Eric Clapton?
- You know, there's a, there's a difference there.
- But the simple answer is more than we have today.
- And if you remember one thing from this presentation, just remember this,
- uh, the industry needs more labeled data than
- we have
- today, and even better so if it's in the public domain where people
- like Skydio and others can use it to build solutions for our
- industry.
- So the, the next slide I'll say is that EPRI is priming the pump here,
- um, for lack of a better term, for AI innovation on
- both intelligent data capture as well as post-processing.
- We're building data sets focused on utility inspection, and then
- we're sharing them under very usable licensing terms, like
- Creative Commons licensing, uh, with, uh, with hopes that people
- like Skydio and others use them to build better solutions for
- our utilities and ultimately the benefit of the public.
- So far, EPRI has shared thousands of images to
- hundreds of entities and
- utilities, private industry, academia, and others, uh, all at no
- cost. And why?
- Is because automation provides the greatest opportunity for
- safe, low cost, and high quality inspections.
- Just like we said on slide one. That's our hypothesis.
- So I'll conclude, um, on the next couple of slides.
- Um, we've performed energized and de-energized lab testing with
- Skydio. We've demonstrated operation in the
- utility wires
- environment, and we've demonstrated inspection
- capability in an
- operational environment. Uh, next month we're actually doing a field
- study to document the inspection speed and quality of using
- an automated, uh, using Skydio's automated approach for
- several
- poles in a distribution right of way.
- I touched on some of our AI dataset work.
- Uh, in the future we hope to, uh, test optimized
- data capture workflows from Skydio and others, and we
- plan to host AI-based competitions to evaluate,
- um,
- post-inspection workflows.
- On the last slide or second to last slide, you'll see my contact
- information here. Uh, reach out if anything's interesting to you.
- I'd, I'd like to thank the utilities that support this research because
- I believe there is power in collaboration.
- And again, that is, that is our tagline here, that
- together we can shape the future of energy.
- So thank you to, uh, everyone that listened in on this call.
- Thank you to Skydio and Commercial UAV News for allowing me to
- present and,
- um, I'm happy to answer any questions.
- Yeah. Real, real quick, uh, 'cause I want to get to the questions as
- well.
- But, uh, I do just want to, again, thank you, uh, Dexter for, um, you know,
- joining
- us and, and sharing, uh, this, uh, uh, information with a broader audience.
- You know, I think our two organizations really well
- aligned.
- Um, you know, for those of you out there
- that don't know, uh, you know, the vision
- statement, uh, of Skydio, uh, is that our mission is to make the world, uh,
- more productive, creative, and safe through autonomous flight.
- Um, you know, so y- leveraging, you know, what we're doing,
- um, and, uh, you know, relying on that adaptive capture
- capability with, you know, robust vision-based
- capabilities, you know, where we can put the system in any
- environment, uh, and be
- able to gather the information that's necessary is extremely important to
- us, you know, and we'll, we'll remain committed to optimizing, uh,
- our products,
- you know, for the utility space, uh, as well as, um, you know, other
- industries, um, you know, where, where drones can, uh, have a positive
- impact. So, uh, with that, I'll pass it back to you, Mao,
- and any questions that we
- might have.
- Thank you, Colin. Thank you, Dexter.
- I also want to echo Colin's, uh, thank you to you for joining us
- and sharing all
- this information. I'm sure everybody's, um, benefiting from it.
- Uh, we don't have tons of time, so I'll get just right to it.
- Uh, there were a couple questions very, very directed to you, Dexter.
- First one is, "What frequency of inspections of the
- infrastructure is currently being, uh, done without
- drones and what frequency do you hope to get with the use of
- drones?
- Is there an optimum frequency of inspection?" So I guess, yes, the question
- is how, how often are drones used, and what would you like to see?
- Um,
- I would say that utilities, uh, that have drone programs
- use drones every day. Um, but they also rely on several
- of their traditional methods. Uh, fixed wing assets for long
- distances, helicopters as well. Um, it really is a
- difficult question to answer because it does depend on the individual
- utility. Um,
- for the most part, a lot of our studies show
- that because of line of
- sight requirements and field travel requirements, the
- business case seems to favor helicopters for comprehensive
- transmission inspection because they can go faster.
- A lot of that, again, is tied to the environment, and
- you can imagine a, a line that's parallel to the road
- is easier to assess,
- access than a line that's in the middle of the woods going through swamps where
- you've got to zig in and out, do your access.
- Um,
- right now the, the business case isn't quite, uh,
- it's not one-to-one, but I think when you begin to leverage in
- autonomy, that's the motivation for doing this work,
- is that it gives you a lot
- more flexibility to do more structures in, uh,
- one flight where you can stand in one location and fly to the extent of your
- line of sight and rely on the aircraft to perform a safe
- mission, getting you the right imagery.
- Uh, that will cut down your inspection time,
- um, and ultimately
- like it, likely make it more cheaper on a, uh, or cheaper
- than a, uh, than a helicopter.
- It just, it's a difficult question to answer-
- It really just depends on the
- utility's inspection needs and what tools they have at their
- disposal,
- uh, to best meet those needs from a safety,
- quality, speed,
- and cost perspective.
- Thank you. Uh, let me just squeeze in one last one, uh,
- before we
- have to hand it back. Dexter, what was the highest voltage you tested with a
- Skydio?
- So those tests were, uh, two hundred and thirty kV line to
- ground 'cause that's what our insulators are rated at.
- And if you, uh, convert that to line to line, it comes out around,
- uh, three hundred and eighty kV, um,
- what, what we did.
- Yeah. I'll, I'll add in there that, uh, this was not in a test environment, uh,
- but, uh, in doing demos, uh, for some other companies, um, you know, we
- have flown, uh, on active lines that were five hundred
- kV, uh, as well as, uh, I, I believe we did one at one point
- that was, uh, up around seven hundred kV as well.
- Um, so you know, not a test environment but, you know,
- just experience with the
- product.
- Thank you, Colin. That's super valuable as well.
- Well, with that, uh, thank you, uh, to both of you for, for,
- for showing up here, taking time out of your day to,
- to cover this content.
- Uh, special thanks to EPRI for, for being a, a great partner
- to Skydio. And, and Laura, with that, thank you
- also to you, and I'll hand it over to you.
- Thank you Mauricio, Colin, and Dexter for a great webinar today.
- Um, many of you have asked about the on-demand
- materials, so just a reminder,
- everyone will receive an email within the next two business days
- that will include, uh, the recording of this live webinar as well
- as a
- PDF of the presentation. So look for that email in the next two
- days. Um, also, if you could please fill out the short
- evaluation
- form that will appear once you close out of the webinar window,
- um, your input would be greatly appreciated.
- And thank you all. Uh, I wanna thank this, our speakers today, and I
- wanna thank all of our attendees for taking the time out of your busy day to
- join
- us. Again, this webinar was brought to you by Commercial UAV News
- and sponsored by Skydio. Thank you all, and have a great rest
- of your day.
Watch Time
1 Hr 2 Min 44 Sec
Related Webinars
2,090 Miles Away. Real Assets. Real-Time Inspection. (Live from DTECH)
Watch webinar
How Cross-Functional Data Powers Modern Utilities (Live from DTECH)
Watch webinar
Why Every Utility Will Have a Drone Fleet Sooner Than You Think
Watch webinar