A Context-Aware Proactive Guiding Approach for Augmented Reality-Assisted Assembly System

Abstract

In the context of intelligent manufacturing, manual assembly remains an indispensable process due to the flexible operations and high cognitive abilities of workers. Augmented reality (AR) assembly utilizes AR technology to overlay virtual information onto the user's field of vision, providing intuitive and immersive assembly guidance on-site. During the AR assembly process, the usability of AR devices and workers’ interactive experience directly impacts manual assembly efficiency and quality. However, existing AR assembly systems can only adopt fixed paradigm guidance due to the lack of comprehensive scene information perception. When user needs change, information presentation is controlled through predefined human-computer interaction commands. This computer-centered passive guidance not only incurs additional time costs but also increases the cognitive load on users, greatly limiting the application of AR assembly.Therefore, this study proposes a proactive guidance approach for AR assembly based on scene information perception. By analyzing the state of parts and operator actions in the assembly scene, the approach can match and update guidance information in real-time. Firstly, a method for context awareness is proposed based on parts detection and assembly action recognition. For parts detection, we create a subassembly RGB image dataset to train the ResNet34 network and use error-proof detection means to improve detection accuracy. For assembly action recognition, we utilize transfer learning strategy by screening samples similar to assembly actions from open-source dataset to pre-train recognition network and fine-tuning the pre-trained network on a self-built target dataset. Secondly, a total-process guidance information modeling method is proposed covering information organization, pushing and visualization. Finally, an AR assembly proactive guidance system is designed and developed. Through a comparative analysis with a passive guidance system, the proposed proactive guidance method is proved to improve assembly efficiency and enhance user experience effectively.

Keywords: Augmented reality assembly, Scene information perception, Part detection, Action recognition, Proactive guidance