We briefly review the rationale and basic principles behind light sources based on (solid state) lasers and luminescence conversion, along with main applications and market need considerations for such sources (examples: medical, information projection, automotive and general lighting). It is clear that the efficiency of high luminance laser pumped phosphor sources depends strongly on the performance of conversion material which is often pushed to optical and thermal limits. Critical materials characteristics and physical mechanisms that control overall source performance will be discussed, also the expected interplay of material characteristics and laser conversion performance. We have separated contributions from thermal and non-thermal quenching mechanisms, and point out the distinction between the true loss mechanisms (e.g. Auger-like or excited state absorption related) and the decline in converted light output due to increased transparency for the pump light (due to activator ground state depletion). The main challenges and roadblocks for improving both the converter materials as well as laser sources are considered and outlook on future discussed: where is the laser based lighting expected to continue from here?