DARPA Taps LSU to Solve Cybersecurity Challenges in Virtual and Augmented Reality
June 20, 2023
LSU researchers Ibrahim “Abe” Baggili and Andrew Webb have received $600K from the Defense Advanced Research Projects Agency, or DARPA, to study safety aspects of virtual and augmented reality applications for U.S. military missions and training.
Baggili’s and Webb’s project, “Mixed Reality Visual Deception for Mission Deviation & Distraction,” bridges cybersecurity with psychology and human behavior to understand the potential effects on people of compromised software and hardware.
“The military is interested in the idea of an augmented warfighter where soldiers can have more information in front of them without having to look at 10 different things at the same time,” said Baggili, who is a professor of computer science and engineering in the LSU College of Engineering with a joint appointment in the LSU Center for Computation & Technology. “But with new technology come new threats, and beyond the technical exploit to hack the system, you could potentially control or influence the person who’s using the system. That’s what we’re looking at.”
Baggili has previously researched how users of virtual reality devices can be forced to move in real, physical space without their knowledge or consent—something he calls a “human joystick attack.”
LSU’s DARPA-funded project will explore the behavior and physical safety of people who use virtual or augmented reality devices or applications, as false, deceptive, or overwhelming amounts of information could be maliciously introduced through a cybersecurity breach. The research aligns with the defense priority of LSU’s Scholarship First Agenda and impacts national security as the U.S. Department of Defense explores the use of emerging technologies for training and military missions.
Webb is an expert on human-computer interaction. His research is primarily focused on improving the design of devices and applications to make them more user-friendly. The DARPA-funded project will explore the physical safety and security of the person who is using devices or applications in virtual or augmented reality—known as VR or AR—often collectively called extended reality, or XR.
While VR presents a wholly synthetic environment such as an immersive video game, AR—also called mixed reality, or MR—layers virtual elements or visualizations on top of a person’s actual surroundings through a lens or headset. MR technology could make it possible to fly a drone or plane with more data available at a quick glance or cross a terrain while seeing a map that displays live updates on people and infrastructure nearby.
“Until now, I’ve mostly looked at these technologies for commercial applications, such as in construction and manufacturing,” said Webb, who is an assistant professor of computer science and engineering in the LSU College of Engineering. “They’re assistive technologies that can help anyone in doing almost any task, so it’s obvious why they’re of interest to the Department of Defense.”
To conduct the research, Webb is building a virtual reality game where the team will collect data on players’ reactions as they encounter false, deceptive, or too much information, which could have been maliciously introduced through a cybersecurity breach.
“This is the first time I’ve looked at cybersecurity as a research direction,” Webb said. “My focus has always been on human behavior around technology, and the wild and interesting things that people do. Here, some of the players will have different levels of experience with XR, which could make them either more or less trusting of the technology.”
“One of the primary areas we’ll be looking at is information overload,” Baggili said. “Can we deviate the players? Can we slow them down? This might not be such a big deal when they’re just playing a game, but what if they’re soldiers deployed in the field, trying to understand and react to a rapid influx of information?”
Professor Ibrahim “Abe” Baggili and Assistant Professor Andrew Webb will lead the project. They’re computer science and engineering faculty in the LSU College of Engineering, while Baggili also holds a joint appointment in the LSU Center for Computation & Technology.
The team will include undergraduate and graduate students who are majoring in computer science or psychology in the research.
“I chose to participate in this project to explore futuristic cyberattacks as new and emerging cyber threats align with current advances in technology,” said Arushi Ghildiyal from Dallas, Texas, who is entering her sophomore year as a computer science student at LSU with a concentration in cybersecurity and a minor in robotics. “This type of experience can be difficult to encounter outside of research.”
Michael Fontenot from Ville Platte, Louisiana, is majoring in psychology and about to enter his sophomore year.
“After exploring different fields in psychology, I came across cyberpsychology, which immediately intrigued me,” Fontenot said. “I reached out to Abe, and then Arushi and I met, and now we’re working on the project together.”
“This is an extremely special opportunity,” Fontenot continued. “I’ve already learned so much about the computer science and tech part, and it’s a huge benefit to me to be this heavily involved in a multi-disciplinary project of this magnitude.”
The subjects for the study will be LSU students, as their age, diversity, and familiarity with technology is similar to those entering military service.
As the project has clear implications for national security, it aligns directly with the defense priority of LSU’s Scholarship First Agenda, which is focused on solving pressing problems for the state and nation.
“The main question we’re trying to answer is, how does visual deception work, and how can we effectively defend against cybersecurity attacks on VR and AR systems to protect our service members?” Baggili said.
“Mixed Reality Visual Deception for Mission Deviation & Distraction” is the second large DARPA grant to LSU in the past decade. The other project, led by Assistant Professor of Mechanical Engineering Corina Barbalata, developed underwater robots to service underwater structures and vehicles.