1. Loss Function : Loss Function  indicates how good a model is or how bad a model is at prediction.
    y = tf.matmul(x,W) + b
    cross_entropy = tf.reduce_mean(
        tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))
    

    In case of digit recognition, to calculate how good or a bad a model  we need actual output ( as represented by y_ ) vs the model output (i.e y). However since in digit recognition the y, model output(y) is represented across 10 binary classes [10]
    i.e model_output_digit_representation = [class_1, class_2, class_3,…… class_9 ] (unnormalised)
    For eg.

    1. 1 may be  [9, 1,2, ….]
    2. 2 = [2,20,4,…]
    3. and so on
      Now, we need to normalise the  (y) model output values  in range [0,1] . This we can do with softmax function. Now once we have the 10 vector (y) model output values  in range [0,1] and 10 vector(y_) values in range [0,1], we can finally compute the loss by simply   taking difference between the two 10 length vectors and summing over it.
      This is done with “tf.nn.softmax_cross_entropy_with_logits”.  Now we need to normalise this loss sum,which we do by taking average. This is done by tf.reduce_mean.
  2. tf.argmax:  Gives the index of the highest entry in the tensor along some axis. In our digit recognition case, it   helps us by converting the softmax values to  index, which  represents digit in our case i.e [0.1, 0.8, 0.2,..] => argmax =>  index 2 ~ digit 2
  3. Depth / Patch :  How many features we want to extract from the patch block i.e
    W_conv1 = weight_variable([5, 5, 1, 32])
    b_conv1 = bias_variable([32])
    

    The weight in the above particular code, will compute 32 features for 5 x 5 patch size. The first two parameters are the patch size,  next param is the input channel and the last param is the output channel.

  4. feed_dict : Used to feed the parameters to the tensorflow graph.
    feed_dict={tf_train_dataset : batch_data, tf_train_labels : batch_labels}

    where tf_train_dataset =  parameter name in the tensor flow graph
    ,batch_data =  data  to be assigned to the  tf_train_dataset

Advertisements