Estimating 3D rotations is a common procedure for 3D computer vision. The
accuracy depends heavily on the rotation representation. One form of
representation -- rotation matrices -- is popular due to its continuity,
especially for pose estimation tasks. The learning process usually incorporates
orthogonalization to ensure orthonormal matrices. Our work reveals, through
gradient analysis, that common orthogonalization procedures based on the
Gram-Schmidt process and singular value decomposition will slow down training
efficiency. To this end, we advocate removing orthogonalization from the
learning process and learning unorthogonalized `Pseudo' Rotation Matrices
(PRoM). An optimization analysis shows that PRoM converges faster and to a
better solution. By replacing the orthogonalization incorporated representation
with our proposed PRoM in various rotation-related tasks, we achieve
state-of-the-art results on large-scale benchmarks for human pose estimation