We have witnessed great technological advancement in Deep Generative models like Generative Adversarial Networks(GANs) to generate state of the art high resolution images in recent years. This has fueled the creation of fake content which is threat to our privacy, democracy and national security. This fake content is referred to as DeepFakes. In this project, we propose a model to effectively classify between real and fake images.The model architecture implemented is Convolutional Neural Network (CNN). The real images dataset is the imagewoof dataset while the fake images are generated by Spectrally Normalized Generative Adversarial Networks (SNGANs) followed by Bicubic, Bilinear and Pixel Shuffle upsampling.
We have used Convolutional Neural Network architecture with three convolution layers, one max polling layer. We have also performed batch normalization. Batch normalization layer is used for normalization and scaling for inputs from previous layer. It makes training of network more faster and stable. The Rectified Linear Unit (ReLU) activation function is used for all the convolution layers. After the convolution layers, we added a fully connected layer at the end. The two nodes in final dense layer in the architecture proposed are used for two finalclasses (real and fake) The sigmoid activation is used at the last dense layer for classification as the sigmoid outputs between 0 and 1. If the sigmoid output is close to zero then the image is classified as fake and if the sigmoid outpout is close to one that the image is classified as real.
The first step in the network pipeline is Data Loading. Both the train and test data are loaded. The real images are assigned the label 1 and fake images are assigned the label 0. Once the data is loaded it is preprocessed to resize it, applied Random Horizon Flip and converted it to a Tensor. the train dataset is split in the ratio of 80:20 to obtain the validation dataset. The model is trained for a maximum of 50 epochs with Early stop implemented to prevent overfitting. The batch size of 40 isselected. The Loss is computed using Cross entropy loss function. For this architecture, we have used Adam optimizer for gradient descent to minimize our loss and update the parameters of our model. We set the learning rate of 0.0001 and the weight decay of 0.001 for optimal training. At the beginning, the model was trained for the entire epoch range of 50. Our training accuracy was nearly 100 percent, so the model was over-fitting. To prevent over-fitting various methods can be used such as cross validation, training with more data, removing feature, early stopping and regularization. we have limited amount of training data and feature removing was not possible, so we have used early stopping. In order to implement this, we have we have defined a parameter patience with value of 2 and trigger time initialized to 0. During training, if the validation loss increase from its previous loss, trigger time increases by 1, when the value of trigger time equals the patience, the training is stopped. Once our Network is fully trained, we now test it with our testing data and compute the loss and accuracy. We then perform the training and testing separately for Bicubic, Bilinear and Pixel Shuffle data and compute the accuracy.
The implementation of this project can be found in the RAML_DeepFake_Detection.py file.
• The model was constructed using a CNN with three convolution layers, increase or decreasein the convolution layers resulted in a decrease in accuracy. • The accuracy of the network drastically reduced when trained and tested for combined datawhen the entire dataset of Bilinear, Bicubic and Pixel shuffle was balanced. (1:1 ratio forreal and fake data). • The network was designed with one fully connected layer with ReLu activation. Addition ofdense layers resulted in decrease in the accuracy. • Adam optimizer provided us with higher accuracy than the Stochastic Gradient Descent.