Second-order nonlinear χ² processes hold the key to realizing various promising classical and quantum applications. Only conventional non-centrosymmetric materials like aluminium nitride (AlN) and lithium niobate (LN) exhibit a strong second-order nonlinearity. While germanium (Ge) has the advantage of mature foundry processing due to its complementary metal-oxide-semiconductor (CMOS) compatibility compared to conventional χ² materials, it has a weak second-order nonlinearity owning to its inversion symmetry. It is predicted that exploiting micro-scale strain gradients induced by silicon nitride (SiN) stressors can break the inversion symmetry of Ge and enable more efficient second harmonic generation (SHG) in mid-infrared (MIR). We herein demonstrate SHG in visible and telecom S-band ranges by pumping our germanium-on-insulator (GOI) sample with a tunable femtosecond pulse laser. The signature quadratic power dependence curve of SHG has also been experimentally observed. In this report, we also propose a novel approach to breaking the inversion symmetry of Ge, which can amplify its weak second-order nonlinearity. We designed a Ge micro-bridge with a periodically changing nano-scale strain gradient that is orders of magnitude higher than the previously proposed structures utilizing SiN stressors. This results in a 3-orders of magnitude enhancement in χ² according to both a classic anharmonic model and density potential theory. Our work paves the way toward a CMOS compatible and high χ² nonlinear material for integrated photonic applications.