Northeastern University researchers have received a $900,000 grant from the National Science Foundation. The money will help them build an AI-powered virtual reality system for students training to work in healthcare.
The project is called the Virtual AI Patient Simulator, or VAPS. It lets students practice talking with simulated patients in settings that look like real clinics.
Faculty from three colleges are working together on the project. They come from the College of Arts, Media and Design, the Khoury College of Computer Sciences and the Bouvé College of Health Sciences.
How the Grant Will Be Used
The award runs for three years. It is part of the foundation's Research on Innovative Technologies for Enhanced Learning program, known as RITEL.
That program supports research using "AI, robotics and immersive or augmenting technologies for teaching and learning," according to the foundation's website.
Eileen McGivney is an assistant professor of art and design and communications studies. She is the principal investigator on the project.
McGivney said one key challenge in health education is giving students contact with unscripted clinical settings. These settings are hard to recreate outside hospitals and doctors' offices.
"We really built it from the ground up," McGivney said. She said the grant "will allow us to build an authoring tool, where clinical faculty who have no experience with virtual reality or AI can create their own patients that are fit for their own curricula and classrooms."
What the Simulator Does
The team is creating custom virtual patients that use realistic speech, eye contact, facial expressions and gestures. Students can practice their communication skills in real time.
The team has already built and piloted the technology. The virtual patient's conversations are shaped by real clinical case records.
Researchers are also using eye tracking and heart rate data from the VR headsets. This data measures how students respond during each conversation.
Leanne Chukoskie, an associate professor in art and design and physical therapy, said a separate animation system is being built. It will make the patients' looks and body language more realistic.
Heidi Cheerman, an associate clinical professor of physical therapy, said the goal is to help students practice "the human side" of care. That includes "communication, empathy, listening."
"What excites us about VAPS is that there is an opportunity to give learners safe environments where they can practice navigating challenging conversations, receive feedback, reflect and try again," Cheerman said.
The project began a couple of years ago with a TIER seed grant from the university. That award helps teams from different fields prove their idea before seeking outside funding.
Sheri Kiami, an associate clinical professor of physical therapy with a background in simulation education, joined after the seed funding. She is helping bring industry best practices into the project, which the grant requires.
"It really will help us overcome a lot of the barriers and obstacles we have in using simulation education to prepare our students for clinical practice," Kiami said.
The team plans to keep testing the platform and build tools to measure student performance. On Oct. 6, faculty showed the simulator in the Immersive Media Lab in Holmes Hall, where physical therapy student Elisabeth Keefner tried the headset.
