Handwritten character recognition (HCR) system recognizes the characters in handwritten papers. The HCR are of two types: online handwritten character recognition, which makes use of machine that collect data in real time, and offline handwritten character recognition that makes use of information from scanned paperwork or pictures. Every person writes differently so offline handwriting recognition is challenging task. Most of the researchers have previously worked in different languages, including Hindi, Kannada, Bengali, Tamil, Telugu, etc. In this paper, we have discussed about handwritten Gujarati vowel recognition along with different methods that are used for feature selection, data pre-processing, data augmentation, and dimension reduction. Finally, we have proposed handwritten Gujarati vowel recognition system on image dataset using deep learning models – Resnet590 and CNN. Both models have been trained and tested with different no of epochs to find the effect of the epochs on the performance of the models. ResNet50 model gives an accuracy values 65%, 71%, 73%, 78%, 78%, 77%, 79%, and 81% for the epoch values 5, 10,15,20,25,30,35,40 respectively. CNN model provides accuracy values 81%, 80%, 86%, 90%, 93%, 91%, 91% and 94% for epoch values 5, 10,15,20,25,30,35,40 respectively.