Self-Supervised Learning for 3D Action Prediction Based on Past Completeness and Future Trend

Abstract

The goal of the 3D action prediction task is to predict the action label corresponding to an incomplete 3D skeleton sequence. Existing studies are limited to the supervised framework. To eliminate the dependence of supervised learning on expensive labels, we propose a self-supervised learning method for 3D action prediction. We use three self-supervised tasks of action completeness perception, motion prediction, and global regularization to allow the network to learn the past and future information embedded in the sequence of unfinished actions, i.e., the action completeness that has occurred and the future motion trend, and to optimize the feature space learned by the model. Some models ignore the past and future information embedded in partial sequences, which is the key to action prediction by humans. Based on our self-supervised method, we design two modules, an action completeness perceptron, and a motion predictor, to complete missing information in partial inputs. And a novel network structure is proposed to fuse partial and complete prediction to achieve more reasonable action prediction. We have conducted extensive experiments on different datasets, and the results validate the effectiveness of our proposed method.