With Benchmark Results

The following is the  benchmark result for the MNIST digit recognition data set as performed across multiple  implementation

Algorithm Training

Size

Training

Time taken

comments
Simple Logistic Regression

(python – sklearn models)

500 0:00:02.361000

(2 sec)

Simple Logistic Regression 5000 0:06:22.674000
6min 22s
Simple Logistic Regression 200000(Full Dataset) Not-computable

(Insufficient memory)

Simple Logistic Regression (TensorFlow Implementation ) 1000 07.845123

07 sec.

Simple Logistic Regression (TensorFlow Implementation ) 10000 54.330695

54 sec

Simple Logistic Regression (TensorFlow Implementation ) 100,000(Half Dataset) 0:08:37.288113

(8 min 37 sec)

 77.0% / 76.2% / 84.2%

Train / Validation / Test Accuracy

Simple Logistic Regression – SGD stochastic gradient descent (TensorFlow Implementation ) batch size 128 05.975 sec  Test accuracy : 86.1%
Simple Logistic Regression – SGD stochastic gradient descent (TensorFlow Implementation ) batch size 12800 4m 12 s  86.5%
Simple Logistic Regression – SGD stochastic gradient descent (TensorFlow Implementation ) batch size 128

num steps : 5001

9s  86.3%
DeepLearning network batch size : 128

hidden size:  1024

01m 45 sec 89.8%
DeepLearning network

(Hidden layer 1)

batch size : 512

hidden size:  1024

8m 10 sec  90.4%

Conclusion :  

1.  Simple Logistic  Implementation with Tensorflow is faster to the simple logistic implementation in python using numpy. This is possibly due to the fact that the TensorFlow  implements the program as graph and takes the whole porgram outside python for faster implementation

  1. Stochastic Gradient implementation  is far much better and efficient to the non stochastic method. this is due to the fact that in stochastic model is trained is randomly selected mini batches, which is more robust and more efficient.
Advertisements