Wide bandgap GaN photoconductive semiconductor switches (PCSSs) have gained recent attention due to high critical electric field strength and high electron saturation velocity to provide high power ultrafast devices. To reduce leakage currents, PCSSs have previously been demonstrated on semi-insulating GaN achieved by Fe compensation doping. However, Fe is known to have strong memory effects, which can redistribute Fe into subsequent films, as well as a narrow window of acceptable doping levels. PCSSs with fast response time are required for applications requiring high repetition rates. C-doped GaN is used to reduce leakage current and improve the photocurrent of lateral PCSSs. The C-doping is performed by simply tuning the growth parameters, such as temperature, pressure and III/V ratio and creates shallow accepter levels, compensating any unintentional n-type doping. In this work, we demonstrate high voltage GaN:C PCSSs with fast response times measured using a picosecond pulsed laser and high breakdown voltages, over 4000 V, measured in a vacuum probe station.