When learning about TensorFlow, I find myself googling for hours, while trying to get things done. I put this exercise here so others may find an example that works, and that includes some things I had difficulties to get done and could not find an example.

So, this example is a simple implementation of handwritten digits recognition using TensorFlow. But unlike other example I found, it uses a data file is from the excellent Coursera Machine Learning course. This file is a subset of the MNIST handwritten digit dataset (http://yann.lecun.com/exdb/mnist) saved as a Matlab data file. All examples of handwritten digits recognition loaded the data like this:

mnist = tf.contrib.learn.datasets.load_dataset("mnist")

And when you get the data differently you may get into one annoying pitfall of getting a cryptic InvalidArgumentError: “assertion failed: [Label IDs must < n_classes] [Condition x < y did not hold element-wise:]...”. This happens because TensorFlow expects the integers from 0 up to the number of classes as class labels (so range(0, num_classes)).

And finally, if you want to use a built-in Estimator, but still you want to use TensorBoard, surprisinly it is very difficult to find how to do that. So this is also included in this exercise. The trick is the run_config usage in the definition of the classifier. And in order to see the graph and summaries, you have to run

tensorboard --logdir=./log

from the same directory you run the script.

Here is the script:

#!/usr/bin/env python3
#
# Simple implementation of handwritten digits recognition using TensorFlow.
# The example allows observing the summaries using TensorBoard.
#

import tensorflow as tf
from sklearn.model_selection import train_test_split
import numpy as np
import scipy
from scipy import io
import logging
logging.getLogger().setLevel(logging.INFO)

np.set_printoptions(threshold=np.nan)

RANDOM_SEED = 0
BATCH_SIZE = 100
TRAIN_STEPS = 5000
LOG = './log'


def load_data():
    # The data file is from the excellent Coursera Machine Learning
    # (https://www.coursera.org/learn/machine-learning) course. It is subset of the
    # MNIST handwritten digit dataset (http://yann.lecun.com/exdb/mnist) saved as a
    # Matlab data file.
    data = scipy.io.loadmat("ex3data1.mat")

    X = data['X']
    y = data['y'].T[0].astype(np.int32)
    # convert all 10s to zeros. Apparently TensorFlow expects the integers from 0
    # up to the number of classes as class labels (range(0, num_classes)). If we do
    # not do this conversion we get the cryptic InvalidArgumentError: "assertion failed:
    # [Label IDs must < n_classes] [Condition x < y did not hold element-wise:]
    y[y == 10] = 0
    return X, y
		

def main():
    X, y = load_data()
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=RANDOM_SEED)
    image_feature_columns = [tf.feature_column.numeric_column("x", shape=[20, 20])]
    optimizer = tf.train.AdamOptimizer(1e-4)
    run_config = tf.contrib.learn.RunConfig().replace(model_dir=LOG).replace(save_summary_steps=100)
    classifier = tf.estimator.DNNClassifier(
        feature_columns=image_feature_columns,
        hidden_units=[400, 100, 26, 100],
        optimizer=optimizer,
        n_classes=10,
        dropout=0.1,
        config=run_config
    )

    train_input_fn = tf.estimator.inputs.numpy_input_fn(
        x={"x": X_train},
        y=y_train.astype(np.int32),
        num_epochs=None,
        batch_size=BATCH_SIZE,
        shuffle=True
    )

    classifier.train(input_fn=train_input_fn,
                     steps=TRAIN_STEPS)

    eval_input_fn = tf.estimator.inputs.numpy_input_fn(
        x={"x": X_test},
        y=y_test.astype(np.int32),
        num_epochs=1,
        shuffle=False
    )

    eval_result = classifier.evaluate(input_fn=eval_input_fn)
    print('\nTest set accuracy: {accuracy:0.3f}\n'.format(**eval_result))


main()