STARE: Realtime, Wearable, Simultaneous Gaze Tracking and Object Recognition from Eye Images
Metadata
- Publisher
- SMPTE — White Plains, NY, USA
- Doc Type
- Journal Article
- Content Type
- Original Research
- Abbreviated Title
- SMPTE Mot. Imag. J
- Volume
- 126, No. 6, pp. 37–46
- Abstract
- We propose STARE, a wearable system to perform realtime, simultaneous eye tracking and focused object recognition for daily-life applications in varied illumination environments. Our proposed method uses a single camera sensor to evaluate the gaze direction and requires neither a front-facing camera nor infrared sensors. To achieve this, we describe: 1) a model-based approach to estimate the gaze direction using red-green-blue (RGB) eye images; 2) a method to recognize objects in the scene reflected on the cornea in real time; and 3) a 3D-printable prototype of a wearable gaze-tracking device. We verify the validity of our approach experimentally with different types of cameras in different illumination settings, and with a proof-of-concept implementation of a state-of-the-art neural network. The proposed system can be used as a framework for RGB-based eye tracking and human behavior analysis.
- Publication Date
- 2017-08-01
- DOI
10.5594/JMI.2017.2711899- ISSN
- Print:
1545-0279| Electronic:2160-2492 - Link
- https://doi.org/10.5594/JMI.2017.2711899
- Author(s)
- Lotfi El HafiMing DingJun TakamatsuTsukasa Ogasawara
- Keyword(s)
- Corneal Image, Eye Model, Gaze Tracking, Object Recognition, Wearable Device
- Copyright
- © 2017 SMPTE
Bibliographic Reference(s)
- J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” arXiv Computer Vision and Pattern Recognition, pp. 1–9, 2016. EXTERNAL
- M. Muja and D. G. Lowe, “Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration,” in Proc. Int’l Conf. Computer Vision Theory and Applications (VISAPP), pp. 331–340, 2009. EXTERNAL
- L. El Hafi et al., “Gaze Tracking Using Corneal Images Captured by a Single High-Sensitivity Camera,” in The Best of IET and IBC 2016-2017, 8:19–24, 2016. EXTERNAL
- D. H. Eberly, “Computing a Point of Reflection on a Sphere,” in The Morgan Kaufmann Series in Interactive 3D Technology, Morgan Kaufmann, pp. 1–4, 2007. EXTERNAL
- J. Redmon et al., “You Only Look Once: Unified, RealTime Object Detection,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 779–788, 2016. EXTERNAL
- H. Bay, T. Tuytelaars, and L. Van Gool, “SURF: Speeded Up Robust Features,” in Proc. European Conf. Computer Vision EXTERNAL
- C. Nitschke, A. Nakazawa, and H. Takemura, “Corneal Imaging Revisited: An Overview of Corneal Reflection Analysis and Applications,” IPSJ Trans. Computer Vision and Applications, 55:1–18, 2013. EXTERNAL
- M. A. Fischler and R. C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Commun ACM, 24(6):381–395, 1981. EXTERNAL
- T. Mertens, J. Kautz, and F. Van Reeth, “Exposure Fusion,” in Proc. Pacific Conf. Computer Graphics and Applications (PG), pp. 382–390, 2007. EXTERNAL
- K. Takemura, S. Kimura, and S. Suda, “Estimating Pointof-Regard Using Corneal Surface Image,” in Proc. Symp. Eye Tracking Research and Applications (ETRA), pp. 251–254, 2014. EXTERNAL
- K. Kunze et al., “MEME: Smart Glasses to Promote Healthy Habits for Knowledge Workers,” in ACM Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 1–1, 2015. EXTERNAL
- A. Plopski et al., “Corneal-Imaging Calibration for Optical See-Through Head-Mounted Displays,” IEEE Trans. Vis. Comput. Graphics, 21(4):481–490, 2015. EXTERNAL
- K. Takemura et al., “Estimation of a Focused Object Using a Corneal Surface Image for Eye-Based Interaction,” J. Eye Mov. Res., 7(3):1–9, 2014. EXTERNAL
- L. El Hafi et al., “Wearable Device for Camera-Based Eye Tracking: Model Approach Using Cornea Images,” in Proc. JSME Robotics and Mechatronics Conf. (ROBOMECH), 2016. EXTERNAL
- M. Geuss, “Why Eye Tracking Could Make VR Displays Like the Oculus Rift Consumer-Ready,” Ars Technica, Jun. 2014, https://arstechnica.com/gaming/2014/06/why-eye-tracking-could-make-vr-displays-like-the-oculus-rift-consumer-ready/. EXTERNAL
- K. Nishino and S. K. Nayar, “Corneal Imaging System: Environment from Eyes,” Int’l. J. Comput.Vis., 70(1):23–40, Oct. 2006. EXTERNAL
- J. Donahue et al., “DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition,” in Proc. Int’l Conf. Machine Learning (ICML), 32:647–655, 2014. EXTERNAL
- C. Nitschke, A. Nakazawa, and H. Takemura, “Display-Camera Calibration Using Eye Reflections and Geometry Constraints,” Comput. Vis. Image Underst., 115(6):835–853, June 2011. EXTERNAL
- L. El Hafi et al., “Model-Based Approach for Gaze Estimation from Corneal Imaging Using a Single Camera,” in Proc. IEEE/ SICE Int’l Symp. System Integration (SII) 2015, pp. 88–93, 2015. EXTERNAL
- L. El Hafi et al., “Gaze Tracking and Object Recognition from Eye Images,” in Proc. IEEE Int’l Conf. Robotic Computing (IRC) 2017, pp. 310–315, 2017. EXTERNAL
Source Data (JSON)
Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2017.2711899.json
Reference Tree
Explore all references and references to this document, as a navigable tree.
Open Reference TreeReference this Doc
Plain text (ISO 690 compliant)
Preview:
Lotfi El Hafi, Ming Ding, Jun Takamatsu, and Tsukasa Ogasawara; STARE: Realtime, Wearable, Simultaneous Gaze Tracking and Object Recognition from Eye Images, SMPTE Motion Imaging Journal ( Volume: 126, Issue: 6, August 2017); SMPTE, 2017. Available at https://doi.org/10.5594/JMI.2017.2711899
Snippet:
Lotfi El Hafi, Ming Ding, Jun Takamatsu, and Tsukasa Ogasawara; STARE: Realtime, Wearable, Simultaneous Gaze Tracking and Object Recognition from Eye Images, SMPTE Motion Imaging Journal ( Volume: 126, Issue: 6, August 2017); SMPTE, 2017. Available at https://doi.org/10.5594/JMI.2017.2711899
HTML (ISO 690 compliant)
Preview:
Lotfi El Hafi, Ming Ding, Jun Takamatsu, and Tsukasa Ogasawara; STARE: Realtime, Wearable, Simultaneous Gaze Tracking and Object Recognition from Eye Images, SMPTE Motion Imaging Journal ( Volume: 126, Issue: 6, August 2017); SMPTE, 2017. Available at https://doi.org/10.5594/JMI.2017.2711899
Snippet:
<span class="citation">Lotfi El Hafi, Ming Ding, Jun Takamatsu, and Tsukasa Ogasawara; <cite>STARE: Realtime, Wearable, Simultaneous Gaze Tracking and Object Recognition from Eye Images</cite>, SMPTE Motion Imaging Journal ( Volume: 126, Issue: 6, August 2017); SMPTE, 2017. Available at <a href="https://doi.org/10.5594/JMI.2017.2711899" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2017.2711899</a></span>
SMPTE's HTML Pub
Preview:
Lotfi El Hafi, Ming Ding, Jun Takamatsu, and Tsukasa Ogasawara; STARE: Realtime, Wearable, Simultaneous Gaze Tracking and Object Recognition from Eye Images, SMPTE Motion Imaging Journal ( Volume: 126, Issue: 6, August 2017); SMPTE, 2017
doi: 10.5594/JMI.2017.2711899
url: https://doi.org/10.5594/JMI.2017.2711899
doi: 10.5594/JMI.2017.2711899
url: https://doi.org/10.5594/JMI.2017.2711899
Snippet:
<li> Lotfi El Hafi, Ming Ding, Jun Takamatsu, and Tsukasa Ogasawara; <cite id="bib-10-5594-jmi-2017-2711899">STARE: Realtime, Wearable, Simultaneous Gaze Tracking and Object Recognition from Eye Images</cite>, SMPTE Motion Imaging Journal ( Volume: 126, Issue: 6, August 2017); SMPTE, 2017 <span class="doi">10.5594/JMI.2017.2711899</span> </li>