An Airbus A350-1000 XWB cockpit.
Thomas Pallini/Business Insider
  • Researchers and engineers at Honeywell Aerospace are working on aircraft systems that can be controlled by voice.
  • Pilots will be able to give commands and have a virtual assistant perform tasks to reduce pilot workload.
  • It's the latest step towards fully autonomous self-flying aircraft that experts say will roam the skies by 2050.

"Hey Siri, fly me to Los Angeles."

Voice-controlled technology is continually advancing and just as Apple's Siri and Amazon's Alexa have seamlessly become part of consumers' daily lives, one company is seeking to do the same with voice-controlled aircraft.

Engineers and researchers at Honeywell Aerospace are currently working on new cockpit systems that will allow pilots to control their planes with voice commands.

It's the latest effort that seeks to reduce pilot workload by increasing automation in the cockpit. The idea is that pilots can give simple commands like tuning to a radio frequency or turning to a heading, while also spending less time on tedious and time-consuming tasks like researching weather.

"Tell me the weather in Charlotte and tell me the critical weather en route," is one command that could be given to ease a pilot's workload, Vipul Gupta, vice president and general manager of Honeywell Aerospace Avionics, told Insider. "And [the] system actually figures it out and displays that information automatically to the pilot."

Gupta likens it to saving time when asking Siri for the weather and having the virtual assistant do all the legwork instead of researching it on a computer. And Honeywell is working to give pilots the same tech in the cockpit as they have in their everyday lives.

Touchscreen technology that emulates smartphones is already common in business jet aircraft like the Gulfstream G500 and G700. Gulfstream's Symmetry flight deck, powered by Honeywell's Primus Epic cockpit, extensively relies on touchscreens to replace instrument panels and traditional flight computer displays.

Gulfstream's Symmetry Cockpit
Honeywell Aerospace/Gulfstream

Even spaceflight has adopted touchscreens, as the world saw when SpaceX's Crew Dragon spacecraft flew Douglas Hurley and Robert Behnken to space. And just as consumers grow accustomed to touchscreen smartphones, they're now becoming accustomed to voice control, and it will soon be another tool available for pilots.

"Alexa has brought that third modality of control into our life," Gupta said of voice control.

Teaching the cockpit to recognise speech

Before voice-controlled aircraft can fly, Honeywell has to teach the system to understand aviation jargon, of which there is no shortage. Telling an aircraft: "maintain 180 knots until ZULAB and fly the ILS 31L approach" is a bit harder than asking for the weather in New York City for next week.

"The eventual goal is to bring in that natural language processing context understanding, and then process it, and then provide the value to the cockpit," Gupta said.

But unlike Siri or Alexa, the response from the aircraft needs to be instantaneous. Pilots can't wait minutes while the system recognises, understands, and executes the command, especially when flying in challenging conditions.

Engineers have refined the response time to 250 milliseconds to ensure the system can move faster than the pilot. Pilot state monitoring through the use of wearable devices will also help ensure pilots remain fully cognisant during the flight and aren't becoming complacent in the flight deck.

Volunteers with a wide range of dialects, accents, and speech patterns come to Honeywell's Deer Valley, Arizona facility and record hundreds of possible commands in a sound booth. In the background while they record are simulated aircraft noises to mirror the sounds of an actual cockpit.

The tech is also tested on a full-motion simulator where pilots can put it through its paces under simulated flight conditions, including turbulence. All the while, researchers are gathering data on how it impacts fatigue levels and whether it actually helps ease a pilot's workload.

Insider demoed a basic version of the tech on a recent visit to Honeywell and it was surprisingly intuitive and quick to respond.

Honeywell's simulator in Deer Valley, Arizona.
Thomas Pallini/Insider

Urban air mobility aircraft and general aviation aircraft will likely be the first to see voice-controlled technology. Single-pilot aircraft are ideal candidates as virtual assistants can help reduce workload.

Electric takeoff and land aircraft, or eVTOLs, will also be primed for more automation due to the sheer number of pilots that will be required to fly them. Honeywell believes it won't be traditional pilots flying eVTOLs, but trained "operators" that will require simplified cockpit systems.

"What will happen is, as you see the growth of the UAM market overall across the world, we will run out of pilots," Gupta said. "There aren't that many pilots out there today who can actually fly all of the airliners, the business jets, and the UAM aircraft. It's just not going to happen."

The goal for most eVTOL operators such as Joby Aviation and Beta Technologies is autonomous operations, where voice-controlled systems will play a similar role. But it won't be a pilot giving the commands, it will be the passenger.

"Eventually, I think we'll have a voice assistant where you will just sit in [the aircraft] and the passenger will say, 'Hey, fly me there, take me there,'" Gupta said. "And then the system does it."

Getting the public comfortable with more automation in aircraft is part of the gradual process of building up to fully self-flying aircraft.

"Voice control is step one - foundational break - then you start building on it," Gupta said.

https://www.businessinsider.co.za/how-engineers-are-working-to-make-voice-controlled-aircraft-reality-2021-5