Industrial robots, which bear the motion for the precision machining of complex parts, have received increased attention from academia and industry. In order to solve the challenges in the realistic reproduction and cognition of the dynamic machining scenes, this study introduces an innovative method called Robo-NeRFCog, which combines NeRF and VLM technologies. This approach enables the 3D reconstruction of robotic dynamic movements and tackles issues in NeRF scene segmentation and action recognition during the grinding process. The NeRF model employs mask technology to separate static and dynamic regions, with the dynamic regions represented by a prior rigid body motion model of an industrial robotic arm as a latent layer. Combined with neural rendering, this yields true 4D rendering effects of the robotic arm across different scenarios. The Qwen2-VL vision language model is used for object category and spatial coordinates recognition in the generated NeRF scenes, and key action analysis in dynamic scenes to study the motion state (Cm) and the effect of specific behaviours—establishing the cognitive knowledge base regarding 3D environments and action execution. The experiments proved that the proposed method was able to improve PSNR and SSIM by around 2% and had reduced the MSE and LPIPS by about 8% from the baseline approaches through the standard evaluation metrics. For the first time, it enables dynamic reproduction under rigid body constraints, representing a significant advancement in the simulation of dynamic robotic motion. This capability offers promising applications in industrial digital twins for guiding robotic motion simulation and machining.
Liu et al. (Sat,) studied this question.