AlGaN/GaN HEMTs grown on GaN substrates have the potential to achieve significant improvement in device performance and reliability. We present a comprehensive study of high performance AlGaN/GaN HEMTs on GaN substrate with an emphasis on correlation of defects with device performance and reliability. Three AlGaN/GaN HEMT structures (AlGaN/GaN-HEMTs-on-GaN, AlGaN/GaN-HEMTs-on-Si and AlGaN/GaN-HEMTs-on-sapphire) were grown and used for comparison. Temperature-dependent electrical performance of AlGaN/GaN HEMTs were analyzed and correlated with defects. The results show that very smooth surface morphology of AlGaN/GaN-HEMTs-on-GaN is demonstrated with significantly reduced structural defects (characterized by HR-TEM) and surface defects (characterized by AFM). Leakage current reduction of more than four orders of magnitude is achieved with AlGaN/GaN-HEMTs-on-GaN structure. The effects of defects on surface passivation and dynamic on-resistance of AlGaN/GaN HEMTs will also be presented and analyzed.