We are working to develop highly-immersive virtual human (VH) interactions that will train health profession students (medical, nursing, and physician assistant) on communication and interpersonal skills. To train these skills, we have built an immersive virtual patient (VP) system that simulates a standardize patient (SP) encounter. These immersive VPs are highly interactive, 3D VHs.
In the 10-minute experiences, the student speaks, gestures, and can touch the VP to practice a patient-doctor interview. Over 450 medical, nursing, and physician assistant students, residents, and physicians have used the system at three universities (UF, MCG, and Keele University School of Pharmacy).
Computer Science inquiries should be sent to Dr. Benjamin Lok (email@example.com)
Medicine inquiries should be sent to Dr. D. Scott Lind (firstname.lastname@example.org)
A medical student interviews an immersive virtual patient
November 2006 – Study @ MCG
Mixed Reality Humans and Intimate Exams
For more information, please visit the Mixed Reality Humans webpage
Aaron Kotranza has led the development of mixed reality humans. A Mixed reality human is a new type of virtual human that allows touch between a human and a virtual human character. We are working to develop mixed reality human patients to allow medical profession students to practice their communication and interpersonal skills during clinical examinations. Preliminary study results show that allowing touch between the student and the virtual patient causes the student to use touch to comfort the patient and to communicate in a more socially engaged manner. Mixed reality humans also enable the simulation of intimate exams, such as breast and pelvic examinations. We are working to integrate mixed reality intimate exams into the educational curriculum of medical students at MCG.
· Kotranza, A., D. Lind, C. Pugh, and B. Lok, “Virtual Human + Tangible Interface = Mixed Reality Human. An Initial Exploration with a Virtual Breast Exam Patient” IEEE Virtual Reality 2008, March 8-12, Reno, NV, 99-106. Best Paper Award, IEEE VR2008 [PDF]
· A. Deladisma, M. Gupta, A. Kotranza, J. Bittner, T. Imam, D. Swinson, R. Nesbit, B. Lok, C. Pugh, D.S. Lind. "A Pilot Study to Integrate an Immersive Virtual Patient with a Breast Complaint and Breast Exam Simulator into a Medical Student Surgery Clerkship." Presented at Association from Surgical Education Surgical Education Week 2008.
After-Action Reviews of Human-Virtual Human Interactions
For more information, please visit the After-Action Reviews webpage
Andrew Raij has led the development of IPSViz (Interpersonal Scenario Visualizer), a tool that enables end-users to rapidly review, evaluate and get feedback on interactions with virtual humans. IPSViz generates visualizations that allow students to review medical communication skills, as well as verbal and non-verbal behavior. Skills and behaviors represented in IPSViz include head gaze, body lean, interpersonal distance, diction, rapport, listening, information-gathering, interrupting the patient, and interview organization.
· Raij, A., and B. Lok, “IPSVIZ: An After-Action Review Tool for Human-Virtual Human Experiences” IEEE Virtual Reality 2008, March 8-12, Reno, NV, 91-98. [PDF]
Racial Bias with Virtual Humans
Brent Rossen is leading studies as to the degree that real world biases (e.g. skin-tone or weight) are transferable to the virtual world. His work his shown that sub-conscious biases are realized in the virtual world and these biases are detectable, measurable, and correlated to psychological measures of bias. For example, in the two virtual humans below, an African-American virtual patient was rated as having lower education and lower income/health-insurance than the Caucasian virtual patient, despite the experiences using the same scripts, animations, audio files, and 3D models (with different textures).
Publications in submission
Immersion and Interface Impacts on Virtual Human Interactions
For more information, please visit the Virtual Patients Software webpage
Kyle Johnsen has led the evaluation of how the display of the virtual human affects the interaction with a virtual human. Specifically, HMDs and monitors significantly alters self-perceptions of the interactions (further from observer ratings) than life-sized fish tank VR displays. Other components that have been studied include audio quality, wizard-of-oz techniques, and physical devices.
Johnsen, K. and
B. Lok, “An Evaluation of Immersive Displays for Virtual Human Experiences”
(short paper) IEEE Virtual Reality 2008,
Soon to be published (Email if you want to be a part of these projects):
Team Members (52 people have contributed to the Virtual Patients project)
D. Scott Lind,
M.D., Professor and Chair of Surgical
Ph.D., Assistant Professor, CISE,
Adeline Deladisma, M.D., Resident, Surgical
Diane Beck, PharmD., Professor, Pharmacy,
Cendan, M.D., Assistant Professor, Surgery,
Chapman, Ph.D., Professor,
Luke Bracegirdle, B.S., IT Development Manager, Pharmacy,
Richard Ferdig, Ph.D., Associate Professor, Education,
Kimberlin, Ph.D., Professor, Pharmacy,
Carla Pugh, Ph.D., M.D., Assistant Professor, Surgery, Northwestern University
Michael Robinson, Ph.D., Professor, Clinical & Health Psychology,
Medical students: Thomas Bernard, Jarrod Craig, Mamta Gupta, Matthew Kalapurakal, Darren Mack, Christopher Oxendine, Hevil Shah, Sarah Szlam, Medical College of Georgia
Jonathan Hernandez, M.D.,
Cyrus Harrison, M.S., CISE,
Joshua Horton, B.S., CISE,
Robert Dickerson, B.S., CISE,
Rebecca Wells, B.S.,
Josephine Albritton, M.D., J.B. Bittner, M.D., Marc Cohen, M.D., James Coverdill, Ph.D.,
Bonnie Dadig, EDD, M.S., Lisa Daitch, PA-C, Margaret Duerson, Ph.D., J. Garrett Harper, M.D.,
Toufic Imam, M.D., Bayard Miller, M.D., Erick Messias, M.D., Ph.D., Robert Nesbit, M.D.,
Mary Anne Park, R.N., Rebecca Pauly, M.D., Brenda Rosson, R.N., Lori Schumacher, Ph.D.,
Patricia Sodomka, FACHE, Amy Stevens, M.D., Dayna Swinson, R.N., Peggy Wagner, Ph.D.,
Sam Wang, M.D.
The following agencies have provided generous support of this work:
Association of Surgical Education, CESERT Grant, 2008-2010
National Science Foundation (NSF Career Development Award and REU Supplement), 2007-2012
Popular Media Coverage
February 17th, 2007 - Augusta Chronicle