2D pose regression using implicit Gaussian mixtures

The task of inferring human poses in 2D image data is known as 2D human pose estimation. Modern approaches to this problem use neural networks to generate keypoint predictions: estimates for the locations of keypoints, such as elbows and knees, associated with the human body. Top-down heatmap-based methods are one such approach, in which predictions are generated for each person in an image independently. This is often done by generating a set of heatmaps which can then be used to predict keypoint locations. Although these methods perform well in practice, they have several disadvantages: the heatmaps they generate are in general not probabilistic and cannot be used to attribute confidence to keypoint predictions, nor correlation between uncertain joint positions. These methods also cannot generally be used in end-to-end regression from input to predictions, as the argmax function is used to determine keypoint locations from heatmaps is non-differentiable. In this thesis, we investigate ways to overcome these drawbacks, with the aim of providing a top-down 2D human pose estimation method which provides predictions which are multi-modal, probabilistic, and differentiable. We accomplish this by creating a model which can generate multiple Gaussian predictions via sampling, leveraging the effectiveness of heatmap-based approaches by generating these Gaussian predictions from heatmaps. Using a sampling procedure ensures predictions are not limited to a small number of preset modes, as is the case with Gaussian mixture models. Implicit Maximum Likelihood Estimation is also used during training, providing a less costly training procedure than performing maximum likelihood estimation on Gaussian mixture distributions. We show that this model is able to generate pose predictions with comparable accuracy to existing methods, while providing rich statistics on the certainty of its predictions over full poses, unlike traditional method which can only produce predictions for each keypoint independently. It is also able to provide multi-modal predictions, capturing information about ambiguous image data that standard heatmap approaches cannot.