Research Paper on ImageNet Classification with Deep Convolutional Neural Networks

1.PAPER SUMMARY:

In this paper they have trained a largest convolution neural network on subsets of ImageNet (ILSVRC-2010 AND ILSVRC-2012 competition) in-order to achieve best error rate results. It was difficult to learn about thousands of objects on millions of images, so convolution neural network model is used because it has a capacity of handling a large data along with the prior knowledge to compensate for the missing data. CNNs are easy to train but are expensive to apply in large scale to high resolution images thus, highly optimized GPU implementation of 2D convolution and all the other operations are inherited in training CNNs. During this training process several unusual new features are added, ReLU Nonlinearity (Non-Saturation Neural model) is used in training the deep learning neural network because it is much faster when compared to traditional saturation neural models. Multiple GPUs are used in training which brings parallelization concept into picture and efficiently reads and writes to one another’s memory directly. Local Response Normalization and overlapping pooling is done instead of the traditional one so that the error rates are reduced. Overfitting is also considered as one of the main issue in training the model in-order to avoid that we use data augmentation and drop out.

2.EXPERIMENTAL RESULTS:

The results on ILSVRC-2010 are recorded based on the experiment. It is shown that the CNN achieves top-1 and top-5 test set error rates of 37.5% and 17.0%5.

Finally, error rates for the ImageNet version from Fall 2009, which contains 10,184 categories and 8.9 million photos. On this dataset, half of the photos for training and half for testing, as recommended in the literature. Because there is no known test set, our split will inevitably differ from prior authors' splits, although this will have little effect on the results. On this dataset, our top-1 and top-5 error rates are 67.4 percent and 40.9 percent, respectively, achieved with the net described above but with a sixth convolutional layer over the last pooling layer.

3.POSITIVE POINTS:

3.1. High Optimization GPUs

Highly optimized GPUs are used to train the large dataset on CNN model without sever overfitting which improves the performance and avoids training time. Training multiple GPUs parallel where they can read and write from one another’s memory directly without using the machine memory which results in the reduction of training time.

3.2. Reduction of overfitting

Overfitting is considered as one of the main issue and using data augmentation by modifying the data I the training set and providing it as the input has reduced the error rate. Dropout is also done where unnecessary layers are dropped from overlapping and backpropagating thus the model uses only required layers to give more accurate predictions and to reduce the error rate.

3.3. Preprocessing the input image

The data set consist of images with different resolutions where they are down sampled to fixed resolution of 256*256.Preprocessing is done by subtracting the mean activity over the training set from each pixel. Thus, the network is trained on the raw RGB values of pixels.

4.CRITIQUES:

4.1. Sensitivity in identifying two appearances in an image.

Suppose if the image contains two apparency present in it such as dog and cherry, the training set has the label cherry but the model determines the label to be dog which is quite confusing and debatable.

4.2. Depth of the network.

By missing any one of the convolution layers will result in the loss of about 2% in top-1 performance, thus the performance of the built model will be degraded.