|
1) Motivation and History
My research interests lay in the area of Human Computer Interaction. I am fascinated with next generation interfaces that go beyond the usual mouse and keyboard human computer interaction.
A substantial research was done in the creation of next generation computer interfaces, but those interfaces can not be applied to the already existing applications. My goal is to create an interface which is usable by an existing application.
Main input for the interface created for this project are eye-movements. The selection of the test application came from the knowledge of the origin of eye-movement-guided interfaces. Eye-guided interfaces are mainly created for the motion impaired people and people with paralysis. I understand that motion impaired people often experience great deal of physical and emotional pain and it would be great to relieve at least some of that. Recent research states that virtual reality can relieve pain in patients. World of Warcraft is a type of virtual reality game with a simple game-play, beautiful immersive world and a rich engaging history. Thus I have selected World of Warcraft as a test application for my interface.
2) Objectives
- Create an interface which will provide an eye- and voice guided means of input to a computer game (World of Warcraft).
- The interface should allow to play the game without the use of mouse or a keyboard.
- The interface will allow to make the game accessible to movement impaired and handicapped people.
3) Equipment
-
The eye movement input is provided by the ASL 504 eye-tracker - device which reads the eye-gaze position on the screen.
- E6600 Core 2 Duo Computer, 2GB RAM, ATI XT 850 PCI-E video card.
- 24 inch wide screen monitor.
3) Main challenges
- Delay (latency) associated with the eye-position capturing and transmission.
- Accuracy of the eye-tracking equipment ~ 1 degrees.
- Noise imposed by eye calibration procedure.
- Eye-tracking errors.
4) Solution
With guidance of my Ph.D. advisor Dr. Javed I. Khan I have developed an Attention Focus Kalman Filter (AFKF).
AFKF is a framework that has following characteristics:
- Noise reduction of the eye-tracking data.
- Improved accuracy of the eye-tracking data.
- Delay compensation through prediction of eye-movements.
The Kalman Filter framework was selected as a rigorous and efficient mathematical tool that estimates the state of a dynamic system from a series of incomplete and noisy measurements. It provided with the means of calculating future eye-speed, thus predicting the area of future attention focus for a particular user.
5) Eye-movement language
The Human Visual System (HVS) is quite complex and consists of several eye-movement types.
As a part of the interface design we would like to design a eye-movement-based language that will provide the means of effective communication between the user and the game.
6) Interface
The current version of the wow percept interface version 1.0 with the source code can be downloaded here.
You need to have World of Warcraft running to use the interface.
I have used VR Commander program for the voice input.
7) Results
This video shows that interaction with the game is possible using eye movements. A character becomes highlighted by green color when it is selected by the eye-movement called an eye-fixation (user looks at a target for a specified period of time). The length of the eye-fixation that triggers the selection of a target is 500 msec.
Notice that my hand don't touch the mouse or keyboard and the selection is done purely based on the eye-movement.
Second video shows me playing World of Warcraft Burning Crusade Beta. Eye-fixations are used to select different targets for an attack. Voice commands are used to move the character around and to pick different spells to attack selected target.
8) Acknowledgments
The work is partially funded by by DARPA Research Grant F30602-99-1-0515.
9) Publications
- Komogortsev, O., Khan, J., Kalman Filtering in the Design of Eye-Gaze-Guided Computer Interfaces, In Proceedings of the 12th International Conference on Human-Computer Interaction (HCI 2007), Beijing, China, July 22-27, 2007. pp. 679-689. [.pdf]
|
