We review the recent progress and perspectives of voltage-controlled spintronic memory devices with ultralow dynamic energy dissipation, for applications in high-performance computing. We first discuss progress in the development of magnetic tunnel junctions using voltage-controlled magnetic anisotropy (VCMA) for switching, which exhibit the lowest power consumption MRAM cells to date (single-digit fJ/bit with precessional switching times < 1 ns). The current device and materials-level challenges and opportunities are discussed next, including biasing, VCMA coefficients, reduced read disturbance, and write error rates. We then evaluate array-level performance of this memory considering different use models within the memory hierarchy and different workloads. Recent results, device proposals and material requirements are then discussed for further reduction of write time, error rates, energy, and bit cell size.