Decoding Cross-Modal Haptic Neural Coupling Through Eeg-Lstm Spatiotemporal Modelling for Vibration-Roughness Interaction

Abstract

Haptic feedback is crucial for enhancing virtual immersion, but a neural coding mechanism that correlates the vibration frequency with the surface roughness in haptic substitution remains unknown. To address this limitation, this study models cross-modal neural coupling between mechanical vibrations and roughness systematically through double-blind experiments, event-related potential analysis and EEG space-time modelling based on the long short-term memory (LSTM) method. By dynamically extracting the spatiotemporal dependence of the EEG signals by the LSTM method and quantifying neural representation similarity using the Euclidean distances, this study reveals that cortical responses activated by specific vibration frequencies are highly consistent with the natural rough perception. The results also show that vibration-touch substitution can simulate rough perception through frequency-tuned neural coding.

Keywords: Haptics, neural coding, cross-modal coupling, tribology