AI and Robotics at TRU: Institutional Augmentation — Part 1

Fink’s vision includes teams of robot dogs on the ground and drones in the air working together to map dangerous terrain perhaps inaccessible to wheeled robots and unsafe to humans, unaffected by smoke and fatigue.

Equipped with sensors such as lidar (light detection and ranging) and multi-spectral cameras, Fink can picture robot squadrons offering a global view, capable of identifying fire intensity and able to crunch enormous quantities of data – with the help of AI – in seconds.

“The name of my lab is perceptive robotics, but if you shorten that off, I call it ‘perro lab’ – and ‘perro’ in Spanish is ‘dog,’” Fink said.

CFJC Today joined Fink for a dog walk on campus.

He controlled one of the Unitree Go bionic canines remotely, prompting it to walk and trot, stand on its hind legs and wave, among other tricks.

Fink can picture the dogs – both autonomous and with shared autonomy – joining BC Wildfire first-attack teams, four-legged friends carrying equipment while working with a drone above to assess surroundings, perhaps mapping safe paths for firefighters.

The dogs (worth about $30,000 each, not including expensive attached payloads) are not well trained enough for that just yet.

Fink said he hopes to hone fundamental science and algorithms and have proof on concept in the next five years, noting longer-legged, more-expensive robots with improved battery life will be required for his vision to materialize.

“From the safety side, it takes a little bit of time for the wildfire… for the world in general to accept robots everywhere, but we can start taking robots into these dangerous places and we can start proving they can work just as good as a human on the ground, but be a lot safer,” Fink said.

BC Wildfire Superintendent of Culture and Prescribed Fire Fons Raedschelders said the sky is the limit for the technology, but it must be capable of integrating into operations and systems.

“Just like helicopters and air tankers, those are people that fly those things,” Raedschelders said. “They’re not just machines. That’s the same with the technology. As it scales, the trick is going to be to find where it fits best and how to integrate it into prescribed fire projects or into wildfire response.”

While the quadrupeds are heeled, not quite ready for the field, drones are already in widespread use.

TRU shares its drone with the Ministry of Forests, whose provincial drone specialist, Matt Sakals, said lidar is useful payload.

“It shoots lazers down at the ground and then receives the information back,” Sakals said. “That way, it accurately maps the surface of the earth. It can map both the vegetation and the ground underneath the vegetation that is difficult or you can’t see with the human eye. That allows us to better understand what may happen after the wildfire or during the wildfire.”

Sakals said lidar can see through vegetation that remains after wildfire.

“We can determine how water and sediment might move as it comes down the hillslopes,” Sakals said. “After the wildfire, we often have natural hazards that occur, erosion and landslides. Understanding better the shape of the earth helps us better understand how the damage may occur or where the damage may occur after the wildfire and then after the landslide.”

Drones designed to drop seed pods can be used to plant trees in remote areas.

“Some of these areas are only accessible by a helicopter, so you would have to fly a person in, drop them in on a rope and have them try to plant some trees on very steep mountain slopes,” Fink said. “That’s where we can have a drone. It can fly in and it can drop various pods and plant the trees that way.”

In the past 10 years, miniaturized computers – such as the Nvidia Jetson – have transformed computing power aboard robots and AI is enhancing data interpretation, including from pictures taken from multi-spectral cameras.

Fink retrains AIs, with the help of biologists, to recognize plants and species, extrapolating data to help further research.

“It can see through things and look in different colours and different frequencies,” Fink said. “It’s a lot better to be able to detect what species we have on the ground. The same species that were in the forest beforehand, are they coming back? Are there invasive species coming back? Are the same trees coming back?”

Raedschelders said BC Wildfire uses drones in weather, fire-edge and hot-spot detection, fire suppression and ignition, ortho-mosaic mapping, fuel monitoring, situational awareness, land management, project intelligence and training.

“There’s been a relationship with fire for millennia with First Nations and Indigenous populations across the globe,” Raedschelders said. “We didn’t do ourselves a favour over the last 100 years, kind of separating ourselves from that relationship all the time. The way to rekindle that relationship and increase knowledge and skills, advance abilities and technology, is through education. We have a lot learning to do and a lot of tools to get there. Those relationships with TRU and UBC are huge stepping stones to that.”

TRU, which operates its Centre for Wildfire Research, Education, Training and Innovation, is using drones in the field to help further research, including in remote burnt high alpine forest in Beaver Valley in Glacier National Park.

“We’re using that data to see how forests are recovering in areas that were impacted by really high severity wildfire [and] in areas that were impacted by low severity wildfire,” said Jill Harvey, TRU assistant professor and Canada Research Chair.

“Does the understory in the forest have a similar composition to before the fire? Are fuels the same? If that forest burns again, what can we expect in terms of the fire behaviour and modelling?”

Students can compare results from what they gather on the ground to what is gleaned by air.

“This presents such an incredible training opportunity for students,” Harvey said. “Students are not only out in the field gaining experience in the backcountry of some of our national and provincial parks using these approaches, but they’re also out there learning about drones, learning about the sensors that we can fly on drones, the type of data that we can collect. And really, these are some of the frontiers we have in terms of natural science and research and also engineering.”

Perhaps Fink’s robot pooches will some day roam remote B.C. forests dropping off the solar-powered modules and sensors built by TRU student Fernando Ferrufino, a network of them reading temperature, humidity and gas, the data then crunched by AI for wildfire-detection purposes.

CFJC Today will take a look at that project and more student creations next Friday (April 24) in Part 2 of AI and Robots at TRU: Institutional Augmentation.

by Marty Hastings

AI and Robotics at TRU: Institutional Augmentation — Part 1

Our Community