It seems like every day I'm writing a new program. I guess that's what getting a PhD in Computer Engineering is all about. We write programs and research their properties. This is in contrast to Psych, or Ed programs that may use software that other people have written to do research. This page is a list of the programs that I have developed during the course of my PhD (well, the 20% that are actually still being used).
I've written (or co-written) a bunch of software for the virtual patients project, all of it is encompassed under the term the InterPersonal Simulator (IPS). The IPS is designed to simulate interpersonal scenarios between a human and a virtual human.
These programs are not directly available to use. If you'd like to work with us on the virtual patients project, please contact my advisor, Benjamin Lok.
IPS_SIMPY - Speech Recognition, Natural Language Processing, Artificial Intelligence, Logging, Wizard of Oz Control (C++)
SIMPY is the brains behind beauty, driving the human-virtual human interaction. It generates appropriate verbal and non-verbal behavior for the virtual humans. It derives its functionality from the literature on expert systems, pattern recognition, and simulation. Also, it allows an operator to have full control over the actions of the virtual human. This is dubbed the "Wizard of Oz" for obvious reasons... The Wizard of Oz is quite useful to handle foreign speakers, and areas of an interaction that are beyond the capabilities of the AI.
IPS_REN - Rendering, Animation (C++, OGRE)
IPS_REN is the real-time graphics engine. It uses the open-source Object-oriented Graphics Rendering Engine (OGRE). In addition to rendering and animation duties, IPS_REN also enables the virtual environment to be displayed across a variety of display devices. Each display devices has its own advantages and disadvantages. Part of my dissertation was on the effect of the visual interface to the virtual human on human-virtual human interaction.
IPS_EDITOR - Scenario Creation (C#)
Editor was my first foray into C# development, and I love it. The purpose of editor is enable scenario creation and maintenance. We currently have 5 scenarios, about 1000 VH responses, and about 100000 sentences that trigger those responses. Currently, Editor only supports script creation, but eventually it will encompass virtual human animation and behavior as well.
IPS_SPEECHREC - Distributed speech recognition (C++)
This is something that we created recently. It's really helpful if you want to have speech recognition in another room. Combined with a wireless microphone, this enables voice training (a somewhat necessary evil) to be done outside the room.
Old: VirtualPatients - The combination of all of the above. We used a library by Haptek for virtual humans animation, and Haptek's full-body characters as the virtual humans. Haptek was actually great, handled physics, lip-syncing, props, animation creation, etc...but...it crashed at random times...that just doesn't fly when you're running studies. We finally ditched it for our own custom-made characters in Maya. It's a lot more work now, but if it crashes, it's our own fault. The problem seemed related to using the c++ library, and not the Haptek Player, so if you're not using that, you might be alright. It's expensive though...$5k for academic license.
If you're interested in Haptek in OpenGL, I did a lot of work to make Haptek characters render properly in OpenGL. I can provide source code to make your life easier.
VTrak - (C++) A flexible, inexpensive marker-based optical tracker
EMagin Z800 Visor Software - (C++) makes using an EMagin Z800 Visor a breeze
VideoAnalyzer (C#) A generic system for coding video. Also implements the Roter Interaction Analysis System (RIAS). RIAS is a system for coding doctor-patient interviews.
VRPN_Recorder (C++)- very nice to have. Hook up a networked (windows) computer to any directshow compliant camera (DV cameras included) and you can programmatically start and stop recording. It is also useful as a standalone video recorder. The big benefit for research is that you can control 20 cameras hooked up to 10 computers all at the same time. Try doing that with a normal video camera program. This program uses the VRPN library (included) and also available at www.cs.unc.edu/Research/vrpn/.
I've written a couple libraries to make working with OGRE easier. These versions work with OGRE 1.25...you need to recompile them to work with the latest ogre
OgreText (C++, modification) I found some 3d text code for OGRE on the web, and I turned it into a library, works well.
OgreOpcode (C++, modification) I modified the OGREOpcode base project to provide the particular submesh you collide with (rather than just a particular triangle). This is useful for selection.
There are two things I find myself needing in every single real time program that I write, timing, and code profiling. Here, you can get two very easy to use classes I've written for these purposes (the timer is independent, the profiler uses the timer). Both are written for windows, although ports should be trivial, provided you have an accurate timer available.
(Disclaimer: All available software is provided as is, no warranty. None of the software is associated with any company)
(Warning: All software is for windows, don't complain)