Abstract/Summary

The ability of the sensory system to create a stable representation of the world from an ever-changing stream of multi-modal information is still not well understood. The aim of this thesis was to investigate the underlying rules the sensory system uses to achieve this in the context of locating objects using vision and touch (haptics). We tested the wellestablished “optimal” combination model (Maximum Likelihood Estimation, MLE) against four other plausible combination strategies for locating objects in threedimensional space. We used a novel methodology that combined immersive Virtual Reality with spatially coaligned haptic robotics and real-world objects. Participants were asked to judge the depth of a target sphere relative to a plane defined by three reference spheres in a two-alternative forced choice discrimination task. A robotic arm was used to vary the depth of the target relative to a plane defined by the reference spheres. Spatially coincident virtual renderings of the spheres were presented on the Head Mounted Display (HMD). Haptic feedback was provided when participants reached out and touched real world objects that were aligned with the virtual objects. The variability of the single modality estimates (vision alone, haptics alone) were used to calculate predictions for performance in the combined-cue condition using five cue combination models. We find that none of the models predict the data well nor is any one model substantially better than the others. Thresholds for the combined-cue condition generally fell between the values of the single-cue thresholds rather than following the minimum variance or MLE prediction. Similarly, biases in the combined-cue case did not fall in the range between those for the individual cues as would be predicted by most cue combination models. The failure of the MLE model in this task has important implications for cue combination theory more widely.