Internetworking and Media Communications Research Laboratories
Department of Computer Science, Kent State University
http://medianet.kent.edu
A Hybrid Scheme for Perceptual Object Window Design with Joint Scene Analysis and Eye-Gaze Tracking for Media Encoding based on Perceptual Attention
Javed I. Khan and
Oleg Komogortsev
Internetworking and Media Communications Research Laboratories
Department of Computer Science
Kent State University, 233 MSB, Kent, OH 44242
Last Revised March 20, 2005
The possibility of perceptual compression using live eye-tracking has been anticipated for some time by many researchers. Among the challenges of real-time eye-gaze based perceptual video compression, are how to handle the fast nature of eye movements with the relative complexity of video transcoding and also take into the account the delay associated with transmission in the network. Such a delay requires an additional consideration in perceptual encoding because it increases the size of the area that requires high-quality coding. In this paper we present a hybrid scheme, one of the first to our knowledge, which combines eye-tracking with fast in-line scene analysis to drastically narrow down the high acuity area without the loss of eye-gaze containment.
Keywords: eye-gaze, perceptual encoding, MPEG-2.
This report contains experiment clips used to test the performance of this system. The videos are MPEG-2 ISO 13818-2 streams. We recommend using MPlayer found at www.mplayerhq.hu or Winamp to view these video samples.
Video Samples: (based on one of the subject’s data)
|
Original Test Videos (bit-rate 10MB/s) |
Saccade Window (for a system with 1 sec delay/lag) |
Saccade Window & Tracking Window (for a system with 1 sec delay/lag) |
Saccade Window & Tracking Window & Hybrid Window Method C (for a system with 1 sec delay/lag) |
Saccade Window & Tracking Window & Hybrid Window Method C (Subject’s Eye-Gazes are displayed) |
Perceptually encoded based on Hybrid Window Method C (1 sec delay/lag; bit-rate 1Mb/s) |
Uniform bit-rate reduction (bit-rate 1MB/s) |