A fusion-based spiking neural network approach for predicting collaboration request in human-robot collaboration

Abstract

In human-robot collaborative (HRC) manufacturing systems, how the collaborative robots engage in the collaborative tasks and complete the corresponding work in a timely manner according to the actual state has been a critical factor that hinders the efficiency of HRC. Inappropriate collaborative behaviors will result in a poor perceptual experience for human operators (e.g., robots starting action too early or too late). To address this issue, a fusion-based spiking neural networks (FSNNs) approach for collaboration request prediction is proposed, aiming to find the right collaboration timing for robots in HRC assembly system and to minimize human aversion without affecting human operation behaviors. By encoding human behavior, product state and robot pose into spiking signals that can be processed by FSNNs, the spatio-temporal coupling relationship between those three aspects can be comprehensively analyzed, and to solve the appropriate timing of robot participation in collaboration. Finally, demonstrative experiments are carried out on the HRC assembly of generator end caps in the lab environment. Compared with the baseline methods, the decision accuracy of the proposed one is improved by nearly 30%, which further proves its effectiveness. © 2022 Elsevier Ltd