SANTA ROSA, Calif. (KGO) -- Robotaxis may be getting all the attention, but robotrucks are already carrying a heavy load of the work at mining and farming sites.
In 2008, Anthony Levandowski accomplished an astonishing feat. He built the first self-driving vehicle to operate on city streets. Now, 17 years later, Levandowski has taken autonomous vehicles off the beaten path with his company, Pronto.
A DRIVERLESS TOMORROW: How Waymo went from secret Google project to dominant robotaxi company
"All the material they make here is what goes in pavement and asphalt," said Levandowski as he surveyed Mark West Quarry on a hill above Calistoga in Northern California's wine country.
The quarry produces construction aggregate like gravel and crushed stone that is used to pave roads.
It's a synchronized operation. Trucks are coming and going along a dirt path, but not all of them have human drivers on board.
RELATED: How this new robot will help build solar farms in a fast, effective way
"The truck is backing up and it is pulling up to the hopper bin. You can see it's autonomous because the blue light is flashing," said Levandowski.
The haulers are being driven by a computer, an autonomous driving system added to trucks and developed by Levandowski's company Pronto.
"We think all mining sites should be autonomous because they are in a dangerous environment," said Levandowski.
Mining is one of the most dangerous jobs in the world. Nearly a tenth of all work-related deaths in the United States are in the mining industry.
Some are catastrophic, like the coal dust explosion in 2010 that killed 29 miners in Montcoal, West Virginia.
But these incidents are rare. More often, it's a smaller accident like a worker struck or crushed by equipment or a truck on surface mines like quarries.
Levandowski said his company can make quarries safer.
A DRIVERLESS TOMORROW: Motional and Zoox take different turns in the autonomous vehicle race
"We actually don't have an operator involved and so there's no way to be injured because there's no person to be there to be injured. The system drives without a person in the vehicle or requires a remote-control operator."
Much of the research and development of self-driving mining machines originated in Pittsburgh, Pennsylvania.
The Steel City, known for its coal mine and industrial origins, has now become a leading hub for robotics.
Much of that can be attributed to Carnegie Mellon University and its long-time robotics research professor, Red Whittaker.
He was finishing up a doctorate in engineering when the Three Mile Island Nuclear Power Plant had a partial meltdown near Harrisburg, Pennsylvania, in 1979.
The incident released radioactive gas into the air.
MORE: Drivers, teamsters want to put brakes on use of automated trucks in CA
With fear of radiation exposure high, Whittaker and his colleagues at CMU built robots that could be controlled remotely to inspect the nuclear reactor. It was then that he realized robots could be used to keep humans out of harm's way.
Early versions of these autonomous rovers moved painfully slow as the computers on board doing the thinking couldn't process the data fast enough.
"These earliest robots would take an image and maybe have to process it for half an hour to make a good plan and to make a good driving action. So one of the big achievements was enough processing for even continuous motion," said Whittaker.
The Terragator was one of the earliest autonomous rovers developed by the newly formed Robotics Institute. It had a camera which allowed it to identify the road ahead.
A DRIVERLESS TOMORROW: How Mercedes beat Tesla to become 1st to offer level 3 autonomous personal car
Soon, Lidar scanners were added, which processed the environment a lot faster, allowing for faster continuous motion.
Eventually, Caterpillar, which manufactures construction and mining equipment, teamed up with CMU to develop self-driving trucks.
Caterpillar's first autonomous trucks appeared in 2013. The company estimates its self-driving trucks have driven so many miles by now that they can circle the earth nearly 7,000 times.
"We are focused on automating the trucks because the task is clear. The truck comes to the loader, receives the material and goes to the spot it needs to and then comes right back and does it again all day long," said Levandowski.
The trucks use signal lights to tell people what the truck is going to do. Amber light means the parking brake is set, a green light means the truck is being manually driven, and a blue light means it is operating autonomously.
Pronto's self-driving kits do not include LiDAR sensors, which measure distances using reflected light. Instead, they rely on global satellite navigation and positioning.
They also rely on computer vision, a set of cameras that use neural learning to identify people and objects in images and videos taken by its cameras.
"We have a front and rear camera on this truck. We are running a lot of machine learning on that. Is the road drivable? Is there debris? Is there someone near? Is there an animal crossing? And so we are able to tell if the area is safe with the camera," said Catherine Culkin, Pronto's Chief Technology Officer.
A DRIVERLESS TOMORROW: Self-driving trucks transport goods on Texas freeways in expansion push, could lead to job losses
Levandowski believes cameras mimic more closely the way humans drive, by identifying objects instead of relying on complex calculations using lasers to measure distance and movement.
"If you look at every single collision with an autonomous vehicle, whether it is our fault or not, and you look at the camera data, it's going to be super obvious to you many seconds before the collision happens just by looking at the camera that a collision is about to happen," said Levandowski.
He thinks what's missing is better software.
Levandowski is one of the godfathers of self-driving technology.
In 2008, he developed PriBot, a Toyota Prius retrofitted with lasers and GPS technology. It was the first autonomous vehicle to legally drive on a city street, going from San Francisco's Embarcadero to the Bay Bridge and onto Treasure Island.
Levandowski later worked for Google's self-driving car program but then left to create Otto, a driverless truck company.
But Otto was shut down after Google accused Levandowski of stealing trade secrets when he left. A judge ordered Levandowski to stop using LiDAR.
Levandowski said he is not prohibited from using LiDAR, he said Pronto does not use it because it is not suitable for a mining environment.
"We need to simplify as much as possible and anything that's on a mining truck is going to break. Trucks are having 100 tons of rocks dumped into them. They shake like crazy. It's a dusty environment. And a solid-state camera is much more reliable than a spinning laser. There's no dust and less to clean," said Levandowski.
He says an autonomous driving system based on cameras is more flexible than one built to rely on lasers.
"We use the cameras to understand where you can and can't drive. And if the world changes because a rock falls or another vehicle is in the way, we would figure out how to navigate around that automatically, not based on a map or something we've pre-made but based on the scenery that's available right on what our software is detecting. The real magic is in the software," said Levandowski.
Besides the Northern California site, Pronto also operates in Brazil and Alaska, in remote areas where there is a shortage of workers.
"By automating trucks, you're able to have the drivers built into the technology and so they're always there, they never leave you, they don't ever get sick and they work really hard," said Levandowski.
That raises fear that machines could replace humans at mining sites.
A study on automation found that nearly half of workers today are at risk of losing their jobs to automation. Levandowski said automated trucks are good for doing the grunt work of carrying heavy loads of rock, but identifying and separating the highest quality rock is still a job for humans.
"When you are loading the material you need to pick some here and there to get the right blend and we think that will need more time to automate," said Levandowski.
But artificial intelligence is pushing the limits. It's being used in tractors to pick out fruit and vegetables, and in wineries to tend the grapes.
Levandowski expects the market for autonomous trucks in mining and farming to keep growing.
"There's about 1,200 driverless mine trucks in the world today, which is more than the self-driving car companies combined and it's still at 1% market share and it's growing exponentially."