My work began by focusing on improving the human-machine interface for bomb-disposal telerobotics. It subsequently followed two paths: improving the display for telerobotics by using stereoscopic video, and improving the input mechanism through application of Augmented Reality.
My early work at the ETC-Lab focussed on the costs and benefits of using stereoscopic displays for teleoperation. As a quick summary of my findings, I would say:
There are several different approaches to achieving Augmented Reality. At the University of Toronto, we combine stereoscopic video with carefully calibrated stereoscopic computer graphics. At the moment, I am investigating the perceptual aspects of Augmented Reality displays. In particular, I am examining the effects of accommodation-vergence conflict and accommodation mismatch on the perception of depth in stereoscopic displays.
Working with Prof. Paul Milgram and Dr. Julius Grodski, I pioneered the ETC-Lab's work in Augmented Reality, culminating in our patented technology which we call ARGOS (Augmented Reality through Graphic Overlays on Stereo-video). One of the first useful examples of Augmented Reality was the virtual pointer, which consisted of a carefully calibrated stereoscopic graphic pointer superimposed on a painstakingly designed stereoscopic video system. The pointer can be moved freely around in the remote view, and used to measure distances and locations of objects in the real world, and can pass this information on to the computers controlling the robots. [SPIE Stereoscopic Displays 1991, IROS 1993]
Most of my effort since developing this technology has been devoted to the usability issues of stereoscopic displays, and the perceptual aspects of Augmented Reality displays. At the moment, I am trying to determine if stereoscopic displays that combine computer generated objects with objects presented via stereoscopic video and objects viewed directly will work. Will people see things where the designers hope they will? Or will distortions, depth cue conflicts, and unforeseen perceptual issues introduce uncorrectable errors?