In real-time quantum feedback protocols, the record of a continuous
measurement is used to stabilize a desired quantum state. Recent years have
seen highly successful applications in a variety of well-isolated
micro-systems, including microwave photons and superconducting qubits. By
contrast, the ability to stabilize the quantum state of a tangibly massive
object, such as a nanomechanical oscillator, remains a difficult challenge: The
main obstacle is environmental decoherence, which places stringent requirements
on the timescale in which the state must be measured. Here we describe a
position sensor that is capable of resolving the zero-point motion of a
solid-state, nanomechanical oscillator in the timescale of its thermal
decoherence, a critical requirement for preparing its ground state using
feedback. The sensor is based on cavity optomechanical coupling, and realizes a
measurement of the oscillator's displacement with an imprecision 40 dB below
that at the standard quantum limit, while maintaining an
imprecision-back-action product within a factor of 5 of the Heisenberg
uncertainty limit. Using the measurement as an error signal and radiation
pressure as an actuator, we demonstrate active feedback cooling (cold-damping)
of the 4.3 MHz oscillator from a cryogenic bath temperature of 4.4 K to an
effective value of 1.1±0.1 mK, corresponding to a mean phonon number of
5.3±0.6 (i.e., a ground state probability of 16%). Our results set a new
benchmark for the performance of a linear position sensor, and signal the
emergence of engineered mechanical oscillators as practical subjects for
measurement-based quantum control.Comment: 24 pages, 10 figures; typos corrected in main text and figure
