Inspecting the Robert Street Bridge with Minnesota DOT and Collins Engineers
The bridge inspection process has changed. The use of automation and drones reduces the need for hands-on inspections, costly bucket trucks, and teams of climbers. In this live talk, the Minnesota Department of Transportation’s Jennifer Wells, Bridge Inspection Engineer, joins us to discuss why they made the shift in 2015 to start using drones for bridge inspection and quickly became an innovative leader in unmanned aerial systems.
Barritt Lovelace, VP of UAS, AI and Reality Modeling at Collins Engineers also joins us, sharing how they partnered with Minnesota DOT to inspect Robert Street Bridge, a historic bridge in downtown St. Paul, and cut their inspection costs by 40%. They walk through how they capture higher-quality data faster, and created precise digital twins using Skydio 3D scan.
Learn more about Skydio for Bridge Inspection.
- Welcome everyone. I'm Matt Bohlinke with Skydio.
- I've been here about 16 months as a solutions engineer.
- And with me today I've got Barrett Lovelace with Collins Engineers,
- and Jennifer Wells with Minnesota Department of Transportation.
- Uh, Barrett, wanna take a moment to introduce yourself?
- Uh, sure. Um, good afternoon everyone.
- Um, I'm Barrett Lovelace with Collins
- Engineers, and I am the Vice
- President of UAS, AI and Reality Modeling.
- But my experience is as a bridge engineer and
- a bridge inspector.
- Welcome.
- Thank you.
- Jennifer?
- Hi, my name's Jennifer Wells and I'm the State Bridge In-
- Inspection Engineer for the administration.
- Um, I started the state in bridge design,
- uh, about 22 years ago now, and ended up in an inspection.
- And run the statewide inspection program for Minnesota.
- Welcome. Uh, thanks both of you for being on the show
- today.
- Uh, definitely have a lot to talk about here
- today about the Robert Street
- Bridge in Minnesota. Um, just wanna give the
- audience a quick cue here. Uh, there is a, a Q&A
- stream off on the right side of your screen, so feel free to drop questions
- in there. Uh, we will get to a Q&A session at the end of
- this
- webinar, and, uh, try to get to as many of your questions as
- we can.
- Uh, so please, uh, feel free to drop those in and, uh,
- we will get to
- those questions. This session will also be recorded
- and available
- for replay on our, uh, website in the next day or so,
- uh, following the webinar.
- Uh, just a quick overview about the Robert
- Street Bridge.
- Uh, this is a structure, uh, bridge that's reinforced
- concrete, um, with multiple arch br- multiple arch
- structure that it spans the Mississippi River in downtown
- St. Paul, Minnesota. Uh, the bridge is, uh, notable for its
- complex design, um, that was required to accommodate the river
- traffic.
- Uh, it also c- has a lift rail bridge, uh, in one
- section, uh, allowing for crossing underneath, uh,
- at an angle.
- And the roadways, uh, on the downtown side of St. Paul.
- Um, this is also a notable bridge, uh, for its
- monumental reinforced concrete rainbow arch,
- um,
- throughout that structure. And wanted to,
- uh,
- give you guys a chance to talk about, uh, your perspective on the bridge,
- and what can you tell us about it? Uh, I'll start off with you Barrett on
- this.
- Sure. So the, the bridge is a pretty important link between
- downtown
- and the west side of St. Paul, and it's, you know, it's getting
- close to 100 years old. It's had some repair and,
- and rehab projects in the past, but,
- um, is, is due for another, um,
- rehab. So our project, um, w- we're doing this
- work for the Minnesota Department of Transportation.
- We're partnered with Michael Baker, and it is to,
- um,
- uh, assess the condition of the bridge in
- detail
- and create a bridge management plan.
- And that bridge management plan will inform
- the next phase, which is a, which is a rehabilitation of the bridge, just to
- make sure that that bridge, um, continues to serve the public for,
- for many generations.
- Awesome. Um, w- what would be the process,
- uh, without drones, um,
- to inspect this? Uh, and how lo- how would that, uh,
- take place?
- Yeah. Um, should, should we... I think we skipped ahead a little bit.
- Oh, sorry. Yep, we did. Sorry.
- Jen, do you wanna do the background for-
- Yep. So for, uh, the Minnesota DOT, a background, um,
- leading up to y- our use of drones in the state.
- Uh, we do have over 20,000 bridges in
- Minnesota,
- both on the state and local system.
- And
- in accordance with our state law, a bridge is defined as 10 feet and over.
- Um, the feds, um,
- say that a bridge is, uh, 20 feet and over, but here we, uh, go
- 10 feet and over. Uh, we started drone research or unmanned
- aerial, uh,
- craft research in 2015, and we took a four-phased,
- uh, approach with this. All of our research, uh, had
- reports done for each phase, and that can be found on the MnDOT
- research website, and it's free for everyone to go look
- at and use, um, as they feel, um, they need.
- No, no need to reinvent the wheel.
- Um, we've done this, um, research in partnership with
- Collins Engineers,
- um, since 2015. We have them under a contract for another
- two years to
- help us with the 3D modeling aspect of it.
- We do have 25, uh, pilots in the state for
- the DOT itself. Um, other, uh, entities,
- uh, on the local system have their own pilots
- as well.
- We do have state aid, uh, positions that help
- the local agencies with that, uh, um, with that pro-
- prospect as well.
- Awesome.
- Uh, you know, this project, uh, is an example of many inspections that co-
- cut
- costs and deliver higher quality data, uh, with the help of
- UAS.
- Uh, could you explain how UAS make a
- difference in the bridge inspection
- field and how you determined that, uh, you could cut costs,
- uh, by upwards of 30%?
- Um, yeah, sure. And, uh, I'll just start with, with our background.
- Um, so, so Collins Engineers, we're a civil and
- structural firm with offices throughout the US.
- Um, we have, right now we have 32 pilots and, and
- 28 drones, and we're sort of known for doing
- inspection of
- structures including bridges, um, which is just one of our
- specialties Um, Collins started by doing
- underwater inspections in the '70s and kind of pioneered that.
- And so the UAS and other technology kind of follows that
- culture of, of having alternate inspection,
- um, methods. Um, and, and really now
- I think,
- um, UAS for us is a pretty routine thing
- that we do and,
- and isn't really a, a novelty or a new, a new thing anymore.
- Um, next slide.
- Um, yeah, so the, the, the traditional way that
- we would inspect this bridge would be
- with, uh, you know, lane closures,
- um, uh, access equipment which, which we're still using, and I'll,
- I'll talk more about that later. But, but generally a detailed
- inspection like this where we're, we're trying to map
- and find every defect on the bridge.
- Um, but it traditionally is just, it's a very manual process and
- includes, uh, you know, maybe we have a copy of the
- plan, some drawings set up. But it, it, it traditionally would
- just be out in the field with pencil and paper and, and doing
- measurements and recording notes.
- Um, maybe doing sketches and taking, uh,
- photographs of different parts of the bridge, um, as part of that.
- Um, but you know, it, it really traditionally has been a
- very manual process.
- Yeah.
- The next slide there.
- Yeah. So, um, this is showing, you know, h- what the...
- what a normal inspection would look like and, and a component of our, our
- efforts here was definitely, um, using some of our traditional
- methods. But really using UAS,
- we can reduce the need for lane closures and reduce the
- time that we're in the field. And, um, and
- in, and also in doing that we can, um, collect higher quality
- data. So we have a lot higher quantity of data
- that's
- available to us at the end, and then, and also we think higher
- quality data. And, and part of that allows us to do
- some, some pre-inspection and, um,
- and we can cut the overall inspection costs and
- inspection time by using these new workflows.
- Yep.
- Uh, Barrett, what would be the process without drones,
- and how long would that
- take, uh, to do this inspection?
- Yeah. So our, our typical process just using pencil
- and paper,
- um, you know, it, it, it would def- it would take probably
- three to four weeks in the field. And, um, we think we probably
- cut that time to about two weeks, two and a half weeks.
- So we probably shaved off a week of time in the field.
- And, you know, we, we're estimating it's about a 30%, um,
- cut in inspection costs and, um,
- but at the same time we're, we're delivering better information.
- Excellent. Yeah, so not only is cost savings, but better information
- overall for, uh, your record keeping and for the customer,
- uh, MnDOT and the public, uh, that they serve. So, excellent.
- For sure.
- Yep.
- All right. Um,
- next slide.
- Go to, um,
- what, what is the pre-inspection work, and how does this impact the
- efficiency of the overall project?
- Yeah. So that, that's kind of the,
- the key to us saving, uh, money or, or being more
- efficient and, and having better data is just, you know,
- the, the... our first step in this whole process
- is to go out, collect all
- of the information with the drones, which is essentially
- images of the bridge and some ground control.
- We create a digital twin of the bridge.
- Um, th- then we take that digital twin and we do pre-inspection
- work in the office before we ever go in the field.
- And we probably, we estimate maybe like
- 70 to 80% of the defects we can find in the
- d- in that digital twin and those images before we ever go
- in the field. So when we, uh... and we, and we take that
- data and we upload it to the, uh, Bentley iTwin
- Experience, which is a, a platform that's on the web that
- we can view these models and organize them and share them.
- And we have these, um, custom inspection forms
- that we create for the bridge. So when the inspectors go out in the
- field, they're, um, they're including the
- annotations right on the model. But the, the pre-inspection work
- really
- helps us because it's, uh,
- we have almost all or about, about 70 to 80% of that
- inspection information already in the mo- model before we go in the field.
- So, so when our staff are in the field, they're much more focused and
- they're much, uh, it's much more efficient.
- They can, uh, you know, a lot of that information they're just
- verifying that it's correct. It's much easier to measure a spall
- or a, or a concrete spall or some kind of defect on the
- computer in the office than it is, it, it is to do that in
- the field. So, um, so it's much more accurate.
- And then, you know, we, we'd still understand
- that being in the field is an
- important part of the process. We still wanna hit the concrete with a
- hammer
- and, and find delams that might be hidden and,
- and those kinds of
- things. But we can add those in the field.
- So we're not really trying to eliminate the, the field work, we're just trying
- to make it more efficient.
- Certainly.
- Uh, Jim, besides efficiency, uh, is MnDOT seeing other
- benefits from pre-inspection work?
- Yes. Um, by doing this pre-inspection work,
- it saves more time
- in the field. Equipment costs to have snoopers out
- there, and lane closure, and all
- this other staff, um, costs a lot of money.
- So by doing the pre-inspection work, we're really reducing a lot of the time
- that you're in the field, um, increasing safety that way by
- limiting lane closure time. Um, also to
- have,
- um, a, a model that we, that they can share with MnDOT staff so
- they can see what is going on, what they're finding out there in, in real
- time as well. Um, so the data quality is just much
- better, and the ability to share this across
- different entities within
- MnDOT is, is very helpful to us.
- Awesome.
- Uh, next slide.
- All right, so this is a quick, uh, quick action
- video of 3D scan, uh, doing exploratory phase on this
- bridge. 3D scan is Skydio's, uh, exclusive
- program, uh, that enables a drone to be able...
- One of our drones to be able to quickly,
- uh, and, uh,
- very organized, uh, develop a flight path for
- the drone to capture
- the ideal ground sampling distance that we want to,
- uh, to achieve, uh, in accuracy. And in
- terms of building a 3D model, it allows us to, uh,
- set that flight path for the drone, uh, in a very highly organized
- fashion, and be able to space those photos exactly
- as we need them,
- uh, to achieve our accuracy level and, uh, the
- closeness up, distance up against the surface.
- Uh, and our grounds, our ground, total ground sampling distance,
- uh, is achieved by, uh, distance from the, the drone from the object,
- and also, uh, by our side lap and overlap.
- As you can see in this, uh, example here, as the drone is moving
- across the surface,
- uh, changing the color from yellow to green to
- blue to purple.
- Uh, of course this video's sped up, uh, very rapidly but,
- um, these, each one of these frames, uh, is where the drone is taking a
- photo, and it's spacing the, those photos ex-, uh,
- in precise order, uh, to allow that drone to capture that, uh,
- great level of detail to get a good 3D model going.
- Um,
- from this, uh, Barrett, uh, what's the process for creating a
- digital twin, uh, for your line of work?
- Um, sure. So, you know, we, we f- first go out and collect all
- these images. We use 3D scan for everything below the
- deck level, and
- then use 2D capture for everything above the deck level.
- So, um, so it, it literally was
- 100% AI and 100% automated flights.
- I think we ended up collecting about 67,000 pictures of the
- bridge or images of the bridge. Um, and then we also collect the groun- ground
- control, and we did it, we broke it up per span
- and, um, processed each span individually.
- But since they have ground control they, we can merge them,
- um, later.
- And, um, so we, we take all of those images,
- all of that ground control, and we process it in iTwin Capture,
- which is a Bentley product. It used to be called Context Capture.
- And we then have a, a digital twin of
- each span. We upload all of that information to the
- iTwin Experience, and then we can toggle off individual
- spans, but we
- can turn them all on and have the, the whole bridge there, um,
- as well. And, uh, and, and Jen kind of alluded to this, that,
- you know, when we have it on the, the web like that,
- everybody on the project
- team or anybody that's involved in the, in the, um, the pro-
- this project or even, you know, going forward, uh, you know, if
- somebody else is gonna do another inspection later for a different purpose,
- we have
- all this historical information that, you know, all you really need is a
- login to be able to access it.
- Outstanding.
- Jen, um, how does, how is MnDOT using digital twins,
- uh, in their inspection of structures?
- Originally, when we started, uh, researching drones, we wanted
- to, to be... We were utilizing it as an access tool
- mainly
- just to, uh, get in places, um, that we can't really
- reach, um, because they were fighting with compliance reviews in
- regards to, um, airness, uh, inspection data.
- Uh, especially in the metro area, uh, we were seeing scope creep
- because bridge inspectors were not allowed a lot of access because
- they didn't,
- in the metro, they don't like to shut down traffic a
- lot.
- It's, it's very hard to get lane closures, um, for the
- state, um, itself.
- So when a contractor goes to bri- a bid on a bridge project,
- they're going off of the inspection data that MnDOT has.
- Um, and because the inspection data wasn't
- accurate as much because
- they're not getting the access, especially to like pier caps and bearings,
- things that such as that, um, it caused, um, issues.
- Um, so a million dollar project would all of a
- sudden the
- contractor's out in the field and, "Oh no, it's,
- there's more problems than what was said on the inspection
- reports." So then the cost then goes up a lot.
- Um, so that's where now the data that we're getting
- from using drones, transitioning from just an access tool,
- but also
- to a 3D model to help with not just inspection projects, but
- maintenance projects and rehabilitation products, um,
- is what is, what we're going towards now, um, to
- use that. And that way we can share it across,
- you know, construction to
- contractors, to load ratings, to bridge design, to,
- um-
- Anything, anything within MnDOT.
- Uh, on any stakeholder, that is
- Sure.
- Excellent.
- It's definitely a, a very beneficial tool for, uh, for both,
- both of
- you.
- Um, just wanna remind the audience, uh, we've got, um, the
- Q&A available off to the side. Feel free to drop any questions
- in for Q&A coming up at the end of our session here.
- We'll be happy to answer your questions.
- Um, we'll go ahead and move on. Uh, slide 10.
- All right. Um,
- Barrett, this question's for you.
- Uh, can you describe, um, um, highlights,
- um, at Collins and MnDOT using iTwin
- Capture for crack and spall detection?
- Sure. Um, so this is, this is maybe one of the most exciting
- parts of, I think, of this whole process, and everybody's, you know,
- c- has, has heard about the AI revolution
- and everything in the news.
- Um, but this is, this is a,
- a way we're using AI, um, now.
- And what we do is we take our digital twins in the
- iTwin Capture software, and we run it through
- the AI process, and it will find cracks for us, and it
- will find concrete spalls for us. And, um, you can see here it
- kinda paints those on the bridge so we can visualize where they are
- and what they look like. Um, but then we can also
- take that data, which includes, you know, lengths of cracks, widths of
- cracks, um, their locations, and we can use
- that for quantities, uh, either for an inspection report
- or for,
- uh, for repair quantities. And that really
- helps us because if we have more accurate quantities, then we don't have scope
- creep like, um, Jen talked about before, and that's,
- uh, a big advantage for us. And, and you don't have to be an inspector
- to, to look at these images and, and understand
- that going out in the field with a tape measure and a
- pencil and paper and trying to map these cracks is almost an
- impossible task. Um, and to, to do it accurat- accurately,
- I think, um, is, is very challenging and very
- time-consuming. And doing it with AI
- takes almost no time for a person. It's just a, it's just the
- computer, um, uh, chugging through and, and
- doing all of this for us. Um, so I think this is something we're
- going to be seeing much more often, and we're, we're using it
- a lot, um, now on, on projects, and even just to be able
- to visualize it like this and see what those cracks look like on a
- bridge. Um, and it also helps us to start looking
- at, you know, how is it changing over time,
- and we can run AI over different data sets to see if the
- cracks are, uh, getting worse or if they're stable, that kinda thing.
- Yeah, that is, that is fascinating, uh, how that AI is able
- to pull those up, highlight them, and, uh, not only that, but you
- have a record going forward of, uh, any advancement
- in those cracks and, uh, monitoring, uh,
- those various, uh,
- uh, fissures there.
- Yeah, awesome. You wanna go to s- next slide?
- Um, Barrett, how has, how has this changed the way you measure
- and detect
- cracks?
- Yeah, so, um, you know, th- this is a pic- just a picture of how
- it would, um, how we're, we're doing the inspection in a
- traditional
- way with expensive, uh, access equipment.
- They're on a, a lift right now. Uh, and they, you know,
- normally we would go out and, and literally just measure the
- crack with a tape measure. Maybe we would mark it on the, on the
- bridge and take some pictures of it.
- Um, but that's a very laborious process and,
- and, and really probably not very, very accurate.
- So,
- so doing it with the, with the AI, uh, just really improves the
- whole, uh, process, improves safety because we're not,
- um, we're not in the field as much as maybe we
- would
- traditionally.
- Yeah. This definitely looks like a very
- dangerous job.
- Uh, these guys are harnessed in and got, uh, their safety gear on,
- um, and obviously on a very high, uh, elevation there from
- the ground. So definitely a, a time saver and
- safety, uh, factor, uh, improvement there.
- Um, Jen, how has, how has this impacted MnDOT's ability to
- maintain bridges?
- So we've been working with, um, Collins and Bentley,
- um, gosh, for the past two years maybe in developing
- this,
- this AI crack detection, especially in the last, um,
- five years or so when the, um, national bridge inspection
- standards have changed. A lot more emphasis has been put onto
- cracking and
- measuring them and, and, and, uh,
- counting them, um, as part of, uh, the element data
- condition states. So having this ability, uh, to
- do that not just in the field but in the office
- using digital twins has really helped,
- um,
- that move forward much faster.
- And also, so we've been working with Bentley for two
- years, providing them
- all, all these different photos that we have in our,
- in our, uh, arsenal to, on different bridges
- and different condition states.
- Um, it's kinda like a learning thing, like how your, your smartphone learns
- how you typically use it over time.
- And, uh, it will, it will learn, um,
- what's a crack and what's not a crack because the user can tell it, "No,
- that's not
- one. That's, that is one." Um, and we're also working towards
- not just crack detection, but also section loss in
- regards to steel-type bridges. Um, going forward too.
- So this is gonna help us a lot, especially to create more accurate data
- and faster data.
- Yeah, that's, that's fascinating, and also, uh, achieving a, a higher
- level of efficiency overall,
- uh, for your agency.
- Um, next slide.
- Barrett, I'll let you, uh, s- expound on this one.
- Sure. So this is, this is a screenshot from the iTwin
- Experience platform. And, um, this is
- showing some of our inspection data on one of those rainbow
- arch, um, arches on the main span of the bridge.
- And on the, on the left you can see the inspection
- information. So that's a custom form that we put together.
- You know, and it includes the, there's an ID for
- that particular defect just
- to, to keep them straight. Um, what's the status?
- You can see that it's been field verified, so they had a box when they go in the
- field to, to check off that they've, they've looked at that
- in the f- in the field even though they, um, probably put it in
- before we went in the field. Um, you know, who was the inspector
- and, um, a d- a description of what you're seeing.
- And then in the middle is the actual model.
- Even though it, it maybe kind a looks like a photograph,
- that is the, the
- digital twin or the model.
- Um, and we can move that around and look at different angles
- and things since
- it's, it's a, a 3D model. And then you can see
- where the defect itself is, is outlined, so that
- gives us the area of that defect. And then on the right are
- the original images of that portion of the bridge.
- So we can just click on that defect or, or an area of
- the bridge that we're interested in and it'll show us all of these different
- images on the right. Um, you know, because the, the images
- probably are,
- uh, better quality than the, the model generally.
- I mean, w- the, the original images are kind of the, the
- highest standard for quality. So we can view those
- images a- along with the model just to see as
- much detail as possible. Um, so all of this
- inspection information kind of resides on this,
- this platform and, and, you know, everybody looking at it right now
- just can quickly understand what's going on and, and you read the
- description. Whereas from a, uh,
- you know, a traditional method it might just be
- notes and you'd have to kind of
- think about, okay, from the description where is this?
- And you'd have to visualize it. But, um, using the
- model and the, the images we don't really have to
- visualize that anymore. We can just concentrate on what we're,
- what we're actually
- looking at.
- So, um, just to recap on this,
- this plays back into your pre-inspection, uh,
- work for the following, uh, year or the following inspection cycle.
- Um, is that
- accurate?
- Yes. Yeah, for sure. And then,
- you know, the, the other benefit of this is
- that the next time we
- have to do an inspection of this bridge we already have all this information
- in there, so we're just adding to it or making changes as things
- progress so it'll make the next inspection even faster.
- Awesome. Yeah, cum- a cumulative library of, of data-
- Yeah
- ... uh, building each, with each, each inspection cycle.
- Yep.
- Awesome. Uh,
- all right. Uh, next slide.
- All right. Uh, Barrett, you wanna touch on this one?
- Yeah. S- so one of the big impacts of, of
- this, th- these new workflows and using the new technology in drones
- and AI and digital twins is, you know, it's, it's having a,
- a impact on our workforce. And, you know, there's some st-
- statistics here, but overall our, our
- industry is, um, getting more funding because of the
- IIJA. We have more need out there, um,
- to, to repair pro- or to, to do repair
- projects and rehab projects. Um, but we don't have an
- increase in the number of staff, we don't have an increase in
- engineers, so we have to find ways to be able to do
- all
- this work with our existing workforce.
- Um, and, you know, all industries are struggling with it, but I think
- particularly transportation. And,
- you know, so this is one way to, to make things more efficient,
- to, to use AI and drones to collect
- data and to process data. Um, and really, you know,
- another thing is, is engineers don't go to, to
- college to, to be stand out in the field and draw
- cracks on a piece of paper. That's not a good use of their time.
- Engineers should be, should be making decisions about
- things and not doing some of these kind of mundane tasks.
- Uh,
- a- and, and so this, uh, helps with that.
- Um, but also I think the,
- we also
- add a component of, like, inclusivity where we can incorporate
- people into the inspection process that maybe wouldn't normally be
- involved, or maybe they can't physically visit the
- field.
- Um, maybe they're in a different location.
- Um, but we can, we can incorporate some of these subject
- matter
- experts into that inspection process. You know?
- And if anything, they can kind of do a virtual inspection
- and
- understand what we're seeing in the field much better.
- Um, and then I think it becomes, uh, at least for Collins, and I
- know we, we've heard this from DOTs, Minnesota and, and others,
- uh, it becomes something that, especially the, the
- younger engineers or inspectors Are really interested in
- using the technology, so it, it becomes a way for us to attract
- staff to s- the inspection part of the,
- of the, um, engineering. And so that's been, been really good
- for us, and it, it makes our jobs more interesting,
- I think, and more fun,
- and, and that's a, a, a benefit.
- Wonderful. Great recap on that.
- Um, next slide.
- Um,
- Jen, um, what's the f- what's the future thinking here, um,
- transitioning to new methodology for, for Minnesota
- DOT?
- Well, we have five key steps to implementing our
- drone
- program. One was the proof of concept test,
- which was our, our phased
- research. Each phase brought in newer technology
- [clears throat] that was being developed. And then, um, going from there.
- We had to start out slow because in 2015
- [clears throat] using drones for transportation purpose was really in its
- infancy. There wasn't any drones out yet
- that could have a camera that
- could look up or utilize without GPS, and that's changed obviously
- quite, quite more, um, over the last eight years.
- Um, but we also needed to be, um,
- careful with privacy issues. You know, we worked closely with legal,
- uh, to reduce bureaucratic red tape.
- We actually had to get permission from the governor in the beginning, uh,
- before we even started to, to do this because we didn't want people
- to see us out there with these drones and get scared that we're, you know,
- filming them or something. And, um,
- so we started out with r- very rural bridges,
- and we worked closely with
- public relations. We did a number of interviews, um,
- with TV channels, n- uh, radio channels, just so that we
- were very apparent about what we were doing to the public,
- um, so that we wouldn't have any issues, and we haven't had any.
- Um,
- that has been due to us being out in the field, just because we have been so
- transparent, and the feedback has actually been very
- positive from the public.
- Um, so also with, um, proof of concept testing was testing
- ways to reduce safety hazards during inspections.
- Anytime you can reduce or eliminate a lane closure by getting
- access a different way is always, is very helpful and
- useful. Um, we did, uh... For one phase of our research, we did, um,
- an actual safety analysis along with a cost analysis to
- show how much we are reducing safety
- hazards in that aspect.
- Now we're at, up in data storage challenges.
- Um, as Barrett had mentioned earlier, there's so much data that a
- drone can collect. So where are we gonna put all this data-
- Mm-hmm
- ... and how do we share it? [clears throat] Our IT department,
- um, a few years back, um, noticed that that was gonna be a
- problem.
- Previously, we had just kept, um, our drone data on separate
- hard drives,
- um, outside of the n- D- DOT network.
- But they created a server or obtained a big
- server just for drone data. So we have it for bridge.
- Uh, photogrammetry has its own folder within
- that server.
- Aeronautics has their own. So right now that's where it's at.
- We're not being charged for it, that, but that could change.
- I know [clears throat] I think
- aeronautics is getting up there with their,
- um, limit. [coughs] Excuse me.
- And so what, that's the next step we're looking at
- because it's still kind of
- clunky. I can't share
- that data with anybody else outside of the DOT, and I'm, I'm
- the owner of the folder and I'm the only person
- that can add people to it.
- It's just, it's kind of a clunky thing
- that we're trying to address.
- Also, along with, um,
- a, a retention schedule. But, you know, we're, we're the state that had a, a
- major bridge collapse, and since then it's all
- bridge data forever. [laughs] Um, so it...
- We're looking at that too, because we don't wanna keep saving data
- over data over data, and then there's gonna be, um,
- crazy amounts of
- data. Um, how is that gonna be useful, become useful then too?
- So finding a way to utilize it and store it correctly, um, is where
- we're at right now. And over the years, we've conducted cost savings
- analysis. That's in our re- our research reports as well,
- um, to show where, um, what bridges in particular
- would a drone help save. It won't save it on ev- on all
- bridges.
- Bigger bridges, um, ones over water typically
- are, are the greatest candidates.
- Um, if there's a lot of trees or brush, that can be an issue.
- Uh, large traffic volumes can be an issue.
- So we do have some parameters that can pick bridges based
- on what a drone would be most useful to.
- Awesome. So a great set of, uh, points to bring up
- here about, um, your cost saving analysis,
- uh, the challenges that you faced, uh, getting your program stood up,
- and then ongoing challenges, as you mentioned, data storage.
- And, um, uh, great job, uh, to DOT
- and to your team, uh, you and your team for getting through that
- and then
- getting this, uh, program up and going. So applause to you.
- Um,
- I wanna jump right in. We've got, um, some time left
- here, uh, and I wanted to jump in on our question,
- uh,
- Q&A. If you guys have a moment, we'll get going with these
- questions here.
- Um, and I think these questions are probably sorted here.
- We could get, uh, both of you, uh, to answer questions here
- that will
- be relevant.
- Um,
- got a, uh, we'll lead off with the first one.
- Uh, what do you do to manage, uh, data at
- scale over time? Are source photos and mesh kept for
- each bridge, uh, for each inspection cycle over time?
- Yeah, that's a, I mean, that's a good question.
- So far we've kept everything.
- Um,
- and it, that, it becomes a,
- it becomes a challenge just because it, you know, it, it can be difficult
- to store all this data. But on the other hand, the data is so
- valuable to us that it also becomes very difficult
- to, to delete it because, you know, we w- we
- went to all the effort to collect it, it becomes historical
- information for us. And, and so keeping it
- is, is important as well so we can start
- looking at change over time.
- I think the, the way we've been addressing it is just
- that, you know, we can share all of these models and
- images, um, on these, on these different platforms.
- We're using the, the Bentley iTwin Experience.
- And that, you know, it's much easier to, to
- save that data on the cloud where, you know,
- Microsoft and Amazon, they're v- very...
- Or Google, they're very good at storing data.
- And, and the prices for storing data has come
- down a
- lot. So, um, I think that's the, the way
- forward versus, you know, it, it can be challenging to s- be storing
- everything locally, on a local network or a local computer, but,
- but being able to,
- um, save it and, and share it even online or on the
- cloud is, is really probably the answer.
- Okay.
- Uh, this one could be, uh, either one of you here.
- Uh, do you have a BVLOS waiver to complete the
- 3D scanning?
- Um, how do you handle maintaining line of
- sight throughout the,
- the inspection, uh, with limited areas for pilots to stand,
- uh,
- and monitor,
- uh, the drone?
- Um, I c- I can answer that.
- Sure.
- Uh, we, we do not have a, a waiver yet.
- Um,
- we... But, you know, on a, on a bridge like this
- one or, or most of the bridges that we're doing, we're
- generally very close to the drone when we're, when we're doing it.
- You know, usually within, um, a few hundred feet of the drone.
- We usually work in teams of two or three, so we can,
- uh,
- we can have an- another person, we have two sets of eyes on the
- drone. And, and so we're, and we're comfortable with,
- with being able to do that. Um, like on this Robert Street Bridge,
- I mean, there probably are times when the drone gets behind a column for a
- second or something, but I don't think that that's
- was the intention
- of the, the, uh, FAA rule that, you know, if it's
- out of sight for a second, I don't think, um,
- um, that's what, what it was designed for.
- We always know where the drone is. We c- we, we have, um,
- two people watching it. Um, but, you know, s- a,
- a waiver like that is probably something we'll pursue in the
- future, especially,
- you know, if you, if you do want to be able to fly from,
- from
- farther away, then that, that can be a benefit.
- Okay.
- Good. Um, another question here from the audience.
- Uh, what ground sampling distance do you aim
- for with flight planning
- for 3D scan feature?
- Um, well, I mean, the closer the better always for
- inspection.
- It's just the, th... And we wanna be able to see cracks
- and things, so we
- generally have to have a pretty
- low, um, ground sampling distance.
- For this Robert Street Bridge, I think we generally were about,
- um, 12 to 15 feet away and, and I don't remember what that
- translates to the, to ground sampling distance.
- But, um, with the, with the sensor in the, in the
- current Skydio, about 10 to 15 feet is,
- is good for inspections, and we can get enough
- detail in there without, you know, h- you know, having,
- um, a, a crazy amount of, of data. But
- even this bridge, I mean, this is a pretty big bridge,
- but it took us less than a
- week to collect all of the data in the field.
- So even at 65, 60, whatever,
- 67,000 images, that really only takes about a week
- for two people to collect. So that's not,
- um, that's not that significant on a project
- of this size
- and a bridge of this size.
- Yep.
- And a, and about, uh, 10 to 15 feet, uh, that's gonna give
- you roughly 1.2 millimeter, uh, there, uh,
- equivalent, uh, ground sampling distance,
- um, with the 70/80 overlap typically.
- Um,
- all right, another question from the audience.
- Uh, have you trained detectors specific to
- your state's photo data
- sets, or are you using stock Bentley
- ContextCapture detectors?
- They're kind of referring back to that AI, uh, ques- um, AI
- segment that we've, uh, talked about.
- We're, we are using our actual photos from our
- bridge
- inspection manual and in our...
- So they are Minnesota photos. They are, um,
- so they're not stock images, they're ac- ones of our actual bridges and elements
- and things like that.
- Okay.
- All right. Uh,
- another question from the audience.
- Uh, how long did it take to capture all
- 6,700, uh, images?
- And also how did you, uh, handle logistics in terms of having
- enough batteries,
- uh, to do your scan?
- Uh, yeah, good question. I probably kind of partially answered
- that.
- So it took, took about a week for two people
- to collect all of the... And it's, it's
- 67,000 images.
- Just-
- But yeah. But the way we did it was we broke it up by span, and
- each span, we probably could do two- To three spans
- per day. Um, and the, you
- know, batteries can be a challenge, but we, you know, we probably
- go into the field with,
- uh, at least 12 batteries, and then we have
- two of the dual chargers.
- And, you know, we used to use, like, inverters or
- generators to charge the batteries throughout the day, but,
- but lately we've been going more towards these battery
- backups like they make. I think Anchor, Anchor makes the ones we
- have, or there's different brands that, that make these battery
- backups that we can charge our batteries.
- And with, with about 12 batteries, two dual
- chargers and one of those, um, battery packs for the field,
- you can fly all day and, and never run out of batteries.
- And, um, and that is important because we don't
- wanna be sitting around
- waiting for, for batteries to charge.
- Um, we'll usually bring an extra drone too,
- and, and we can even,
- on a bridge like this, I think there were times when we were flying two drones
- at the same time. And it's re- it's really interesting how it's
- changed too over the years because when Jen and I first started doing this,
- you'd go in the field and you'd be very nervous
- flying around the bridge and f- afraid you're gonna run into something.
- And now it's gotten to the point where you're almost bored
- doing it because the drone is doing all of the work,
- and you literally
- are just waiting for the drone to, to finish.
- [laughs] So you start to,
- to, to kind of get, um, bored with what you're
- doing. Um, but I think that that's a sign of how much
- the 3D scan and AI has really helped us because w-
- we don't worry about crashing anymore and we, um, you
- know, we, we can get in all these really tight areas
- with the
- drone.
- Yep. Do, uh, another question from the audience.
- Do you fly over live traffic while gathering the data, uh, or
- is the traffic restricted on the bridge or cordoned off in a-
- another area?
- Um, that, that's a good question. The...
- For this particular bridge, we, um, you know, we didn't
- fly over traffic, but we never closed
- the traffic on the bridge to do the scans either.
- Um, it's, you know, it's a f- it's a fairly busy
- bridge, but,
- um, but there's, you know, there's stoplights on either side of the
- bridge, and so
- there's breaks in traffic. And what we would do is, you know,
- we'd be up on the
- bridge while we were doing it, um, when we're flying over the top of the
- bridge,
- and we're flying our, our missions, and we just watch for traffic.
- And when cars come, we'd pause the drone
- and maybe move it out of the way
- so it's not over traffic. And, and then we can
- resume when, when traffic is, is back, uh, under the
- bridge. And that works for,
- uh, almost all the bridges we do. If we, if we get a bridge where there's really
- heavy traffic, we'll either fly it from the side and, and
- take images at an angle to get the deck, or we
- can also have lane closures to, to do that as well.
- Yep. Yeah, capturing those oblique angles, uh, definitely
- adds a element of, of dimension in there, but plus
- coverage without having to shut down traffic.
- For sure.
- Yep. Um, another question from the audience.
- Uh, this was, uh, appears to be a in controlled airspace.
- Uh, what clearances or waivers, uh, were required?
- Um, yeah, so the, uh, home and field, the, the St.
- Paul Airport is, is pretty close to here.
- I think our ceiling was,
- was, like, 100 feet, but we just have to go through that, the,
- uh, airspace waivers and use the app, and we can get immediate
- clearance for, for flying at this bridge.
- So that was, that was pretty easy, and that's pretty...
- That's a big improvement over, you know, where we were five or six
- years ago. Um, you know, to be able to get immediate
- authorization in, in an area like this, um, is, is
- nice. But we would have to do that every day we went out to fly.
- Um, got another question from the audience
- here.
- Have you found any value in immersive inspection,
- um, uh, in paren, augmented reality
- versus solely using a computer to inspect the model?
- Um, yeah. So Jen and I have both, with Bentley, we've
- worked on
- some different platforms to use AR and VR to, to view the
- bridge. And, um, and, you know,
- we've used the Microsoft HoloLens.
- There's a whole platform from, from Bentley that you can use for
- inspection. And I think it is a really interesting
- application and the, you know, the...
- You get a much better feel for what,
- what it like, what it's like to be at the bridge using
- VR,
- AR. Um, and right now we've been
- working on, uh,
- um,
- uh, some VR stuff with the, with the Quest headset
- and with, uh,
- uh, Unreal Engine, which, you know, if you have kids that play
- video games, you know what that is.
- But we've been working with Bentley on a, a platform to,
- to view it in, in VR. And, and that's where it,
- it really starts to feel like a virtual inspection.
- And we can, you know, our designers can
- sort of visit the bridge in a virtual way and get a better understanding
- if they're doing a load rating or trying to design some repairs or,
- or those kinds of things. Um, even for, like, public involvement,
- uh, that's a, that's a good application.
- Um, and for training, you know, for training bridge inspectors
- or just even to, you know, like for, for the bridges that we
- have already created digital twins, just to be able to plan the next
- inspection and, and view it in that manner for, for planning
- purposes is a big benefit as well.
- Awesome.
- Yep.
- Um, another question from the audience.
- Uh, are you able to identify hollow areas in
- the
- concrete using AI detections, so a void,
- uh, behind the surface?
- Jen, do you wanna answer that?
- No, we're not able to do that. [laughs]
- [laughs] X-
- No, there's... We, we don't have a Superman camera yet.
- No. [laughs]
- The x- X-ray vision hasn't come out.
- That-
- Right
- ... that's, that's the next model.
- Yeah.
- Yep.
- There, there are some ways to do it with-
- Yeah
- ... with infrared.
- Yeah.
- But that's, uh-
- It's hard
- ... yeah, it's hard.
- Okay.
- Understood. All right. Um,
- lastly, I wanna touch on, uh, for both of you, um,
- uh, you guys use a variety of drones.
- In your experience, uh, what, what has been the most,
- uh, useful platform for your workflow,
- um, here in the recent years?
- Yeah, I think for us, you know, we've...
- We started off using, like, a SenseFly eBee all the way a long time
- ago.
- We have some Falcon 8s. Um, w- as a company,
- we've, we decided to,
- to only buy American drones just because of our, our
- clients and things, so we're probably 90%
- Skydio drones now. And, um,
- and especially for a bridge like this, the...
- They're just what... There isn't another drone that would,
- would work for this.
- There's, there isn't another drone that can get in those tight places
- and be able
- to collect the data on such a, a complex bridge
- like this, um, with, with a different drone.
- Um, so we, we mostly have been using
- Skydios, and most of our offices throughout the
- country
- have at least one Skydio drone. And that's also
- been,
- uh, nice just because,
- you know, with the obstacle avoidance, it's much,
- it's much more approachable for engineers.
- And we,
- you know, i- in our company at least, we don't have people who are drone pilots.
- We have engineers and inspectors who use drones for their
- work. And so the, the Skydio drone has worked well for
- that because it just kind of builds confidence with people
- with the, the, um, 3D scan and the,
- the obstacle avoidance.
- I love that. Uh, you don't have to be an expert to fly it.
- And it's, it's another tool in the toolbox for each
- one of your
- engineers-
- Mm-hmm
- ... uh, to be able to do their job. Uh, Jen, do you...
- Can you expound on, uh, your drone program and, uh,
- uh, describe your drone program for us?
- Yeah. Um, we got a grant from Federal Highway
- in 2018. Um, and at that time, uh, they,
- we got a bunch of DJI drones, um, which work great.
- They fly really nice. Um, birds seem to like them, though.
- [laughs] I've had a, a couple of bird attacks from
- that, that, that particular drone.
- Um,
- uh, but,
- uh, our state aid, uh, personnel just recently bought,
- uh, two Skydios, and they're gonna buy another three.
- And we are going after one of those grants, uh, smart grants, uh,
- coming up here in October to, uh, purchase more
- Skydio drones because they're way more conducive to, um,
- structure inspection like this because of the obstacle avoidance, because,
- um, the GPS doesn't get confused, um, like with the...
- We've had some of our DJI Manics- Mavics, um, get w- uh,
- a little wily around steel especially, and lose
- signal and, or just go fly off somewhere.
- So, um, we're tran- transitioning from those
- to, you know, American-made products obviously 'cause
- it's so much easier,
- um, especially when we have, uh, a drone that goes in the drink or,
- or has an issue. It's much easier dealing with, um,
- within our own
- country with that. Um, plus because just the obstacle
- avoidance, uh, is, is good too. Um, and they're easy to fly
- so. We also have, um, uh, one of those cage drones, the
- Elios. Um, that thing's really hard [laughs] to fly.
- Um, it's made for confined space, and we do have a lot of hollow
- bridges, uh, that are in confined areas, but that one is very, very
- difficult to fly. You have to be pretty skilled to fly
- that one well.
- So yeah, we're... So the D- you know, the DJIs were a great, you know,
- starting point for us to learn and, and get better with our
- skills in flying, but we've had a number of,
- um,
- demos with Skydio, and it's just way...
- They're more,
- way more conducive, like I said, to structure inspection for us.
- Very good.
- Yeah, I, I feel like our, uh,
- our
- shared challenge here is getting, uh, Skydio in, in
- engineers' hands and, uh, in the work...
- Built into the workflow that you're doing.
- And, uh, through that you've been able to, uh, outfit
- your units with
- easy to fly, uh, hard to crash drones, and Skydio is the
- way, uh, I think that, uh, is gonna help, uh, get that mission
- accomplished, uh, get you the ground sampling distance
- that you need to capture
- those images. And, uh,
- I look forward to our, uh, long-lasting, uh, relationship and getting,
- getting these inspections going forward so.
- Um, we're r- about out of time. I just wanted to...
- Barrett and Jennifer, I wanted to thank you both for, uh,
- your time
- today and being with us. Uh, for the audience, uh,
- we weren't able to get to everyone's question, but if you have
- additional questions, feel free to visit our website,
- skydio.com, and, uh, you can obviously drop a question in
- there. We'll be happy to reach out to you, uh, uh, as soon
- as we can get, uh, an answer for your question.
- So, thank you everyone for your time.
- Um, Barrett and Jennifer, thank you again both, uh, for your time
- as well today.
- Yeah, thanks for having us.
- Appreciate it. Yeah.
- Thank you.
- It's been a pleasure. Yeah. Thank you all. Have a good day.
Watch Time
55 Mins 56 Secs