Earth was hotter in its early history, though few observations exist on the cooling rate of the bulk Earth. We demonstrate that rates of true polar wander–wholesale rotation of mantle and crust around the core–have significantly decreased over the past three billion years. As true polar wander is rate-limited by the viscosity structure of the mantle, we interpret the decay to reflect the secular cooling of Earth. The amplitude of the decay indicates that the viscosity of the lower mantle had increased by an order of magnitude, and such a viscosity requires a cooling rate of >100 K/Gyr for the lower mantle. This rate is compatible with recent petrological and geochemical estimates on the cooling rate of the upper mantle, suggesting that the mantle as a whole has experienced rapid cooling since at least Neoproterozoic time, and possibly since Archaean time.