Experiment Three Mind as Mouse
This is experiment three out of the human-scaled natural interaction series that explores the mind and eyes as a navigational mechanism.Having explored proximity and light for human scale input, we asked ourselves if there was another way of looking at interaction, beyond the physical space. Like our previous experiments, we weren’t aiming to invent a new method of input; but rather seeking to explore non-conventional input for natural interaction. We asked ourselves, what could be more natural than recognizing and enabling control with the cognitive focus of the individual interacting with the system? We wondered if eye-tracking fit into our human scale theme, but ultimately we agreed that we had found an important inflection point for our thinking around the subject. What could be more human scale than working at the level encompassing human perception, with a direct link to the mind
Test 1: Poor man’s Research
Although we were interested in eye-tracking for interaction purposes, as a company engaged in Design and Research, we would have been remiss not to consider application of our technique for concept testing and other forms of research. Most eye-tracking solutions are expensive and awkward for the participant. By adapting our camera-input technique to focus on the dark point of the pupil, as opposed to the bright point of an LED, we had an inexpensive (requiring only a web-cam) and non-invasive method for tracking visual focus.The first problem we encountered was that of isolating the pupil. We overcame this by constructing a light-box (out of a cardboard box), inside which we suspended a display and camera. When a person peers through a hole on the side of the box, we surround the eye with a well lit surface, ensuring that the pupil would be in fact the darkest shape that the computer sees.
 |
 |
Of course, our box is only a temporary solution. Future implementations might overcome this problem in software during calibration, or by at least reducing the lit surface to a simple stand in place of a light box.Here [indicating an image inline] we draw the path of the eye in an example study of our own website, indicating the timing and order of viewing by transitioning through the colors of the light spectrum. We chose this implementation for our test as an alternative to the more common heat-maps that we have seen, because we wanted to produce a visualization that allowed the researcher to immediately discern the timing and order of focus at a glance.
Test 2: Magnifying and Revealing Focus
The first interaction concept we explored was to zoom in on the focal point which we were tracking. To do this we created a small gallery of photos and simply enlarged images when they came under visual focus. This worked quite nicely. Next we created a menu with large target areas. As visual focus falls on an individual item we would apply a tint and glow to the item, revealing for everyone other than the person operating the prototype which item in the list it is that they are focusing on.
 |
 |
One limit that we found with our technique (specific to using a web-cam) was that by calibrating the input in on the area of the eye we lost a great deal of resolution, and the resulting patterns in tracking were forced onto the grid of pixels produced by the bitmap rendering of the digital image. Of course, the solution for this would be to use lens-based zooming to maximize resolution before calibrating.
Envisioning real-world applications
We see this kind of affordable eye-tracking becoming a real possibility if vision-based computing arrives as an input for natural interaction. Commercial applications range from design research and analysis to computer systems which augment gesture and touch interaction with a genuine understanding of the user’s visual focus. One idea in particular that stood out in particular for us was that of medical applications which could enable communication between patients who are physically mobilized and unable to speak with their families and hospital staff.