Spiking recurrent neural networks are gaining traction in solving complex temporal tasks. In general, spiking neural networks are resilient and computationally powerful. These intrinsic properties make them attractive for learning on the edge of the devices. In this work, we propose a semi-supervised deep spiking neural network that can be deployed on embedded devices. We present memristor synapse and neuron circuits integrated in a neuromemristive system to implement a deep spiking neural network. For optimal performance on-device, the synaptic crossbar array is trained using a modified gradient descent rule in conjunction with an unsupervised rule. The proposed deep spiking neural network is validated for on-device video classification tasks.