Using a high terahertz electric field (E_THz), up to 282 kV/cm, the optical conductivity of monolayer graphene is studied as functions of grain size and doping concentration. In order to study the effect of the grain size on the optical conductivity, the graphene with three different grain size, small (5 μm), medium (70 μm), and large grains (500 μm), are prepared by controlling surface morphology of Cu substrates. The dominant carrier scattering source in large- and small-grained graphene differs at high field, i.e., optical phonon scattering for the large grained, but defect scattering for the small grained. The enhanced optical phonon scattering in high THz field from the large-grained graphene is caused by a higher optical phonon temperature, originating from the slow relaxation of accelerated electrons. Unlike the large-grained graphene, lower electron and optical phonon temperatures are found in the small-grained graphene, resulting from the effective carrier cooling through the defects, called supercollisions. In addition, the optical conductivity of graphene monolayer shows an opposite behavior between at high and low doping concentration of the graphene. At large Fermi energy > 110 meV, the conductivity reduced with increasing THz field, resulting from the emission of optical phonons by excited electrons. However, for ≤ 110 meV, electron–phonon scattering rate is suppressed owing to the diminishing density of states near the Dirac point. Therefore, E_THz continues to accelerate carriers without them losing energy to optical phonons, allowing the carriers to travel at the Fermi velocity. This leads to the enhanced optical conductivity of graphene when E_F is low. The exotic carrier dynamics does not result from the massless trait, but the electron-optical phonon scattering rate that depends on Fermi-level in the graphene. Our observations provide insight into the application of graphene for high-speed electronics without degrading carrier mobility.