Augmented reality (AR) assembly guidance systems can help users quickly master the assembly process of unfamiliar objects. However, it is very difficult for practical use without a thorough understanding of users' intentions. Furthermore, existing approaches still struggle to handle complex hand-part interactions and nonlinear assembly steps, and achieving intuitive and real-time guidance remains challenging. To address these issues, we propose an intention-driven camera-projector AR assembly guidance system (IntentionAR) that integrates online user intention inference with a finite-state assembly machine. The intention module recognizes interaction actions and infers the target object, enabling feedback before assembly begins, while the finite state machine (FSM) manages step progression. Using a camera-projector device, the system highlights candidate parts and, based on inferred intention, flags correct/incorrect selections to provide real-time guidance. As the assembly progresses, the models used for spatial registration and visualization switch dynamically to accommodate the nonlinear workflow. Experiments and user studies show that the system delivers a more robust AR interaction experience and improves assembly efficiency.
Cao et al. (Thu,) studied this question.