GXNOR/MNIST inference

The MNIST dataset is a handwritten digits database. It has a training set of 60,000 samples, and a test set of 10,000 samples. Each sample comprises a 28x28 pixel image and an associated label.

This tutorial illustrates how to use a pre-trained Akida model to process the MNIST dataset.

1. Loading the MNIST dataset

# Various imports needed for the tutorial
import os
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
import warnings
from tensorflow.keras.utils import get_file
from tensorflow.keras.datasets import mnist
from sklearn.metrics import f1_score, accuracy_score

# Filter warnings
warnings.filterwarnings("ignore", module="matplotlib")

# Akida specific imports
from akida import Model
# Retrieve MNIST dataset
(train_set, train_label), (test_set, test_label) = mnist.load_data()

# Add a dimension to images sets as akida expects 4 dimensions inputs
train_set = np.expand_dims(train_set, -1)
test_set = np.expand_dims(test_set, -1)

Out:

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz

    8192/11490434 [..............................] - ETA: 0s
 1196032/11490434 [==>...........................] - ETA: 0s
 3506176/11490434 [========>.....................] - ETA: 0s
 4513792/11490434 [==========>...................] - ETA: 0s
10133504/11490434 [=========================>....] - ETA: 0s
11493376/11490434 [==============================] - 0s 0us/step

2. Look at some images from the test dataset

# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
    axarr[i].imshow(test_set[i].reshape((28, 28)), cmap=cm.Greys_r)
    axarr[i].set_title('Class %d' % test_label[i])
plt.show()
../_images/sphx_glr_plot_gxnor_mnist_001.png

3. Load the pre-trained Akida model

The pre-trained neural network model is available on Brainchip data server You only need to pass this .fbz file to the Akida Execution Engine in order to instantiate the model.

# Load provided model configuration file
model_file = get_file("gxnor_mnist.fbz",
                      "http://data.brainchip.com/models/gxnor/gxnor_mnist.fbz",
                      cache_subdir='models/gxnor')
model_akida = Model(model_file)
model_akida.summary()

Out:

Downloading data from http://data.brainchip.com/models/gxnor/gxnor_mnist.fbz

  8192/373501 [..............................] - ETA: 0s
 73728/373501 [====>.........................] - ETA: 0s
270336/373501 [====================>.........] - ETA: 0s
376832/373501 [==============================] - 0s 0us/step
                        Model Summary
______________________________________________________________
Layer (type)                 Output shape  Kernel shape
==============================================================
conv_0 (InputConvolutional)  [14, 14, 32]  (5, 5, 1, 32)
______________________________________________________________
conv_1 (Convolutional)       [7, 7, 32]    (5, 5, 32, 32)
______________________________________________________________
conv_2 (Convolutional)       [7, 7, 32]    (5, 5, 32, 32)
______________________________________________________________
dense_2 (FullyConnected)     [1, 1, 512]   (1, 1, 1568, 512)
______________________________________________________________
dense_3 (FullyConnected)     [1, 1, 10]    (1, 1, 512, 10)
______________________________________________________________
Input shape: 28, 28, 1
Backend type: Software - 1.8.4

4. Classify a single image

Now try processing a single image, say, the first image in the dataset that we looked at above:

# Test a single example
sample_image = 0
image = test_set[sample_image]
outputs = model_akida.evaluate(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % test_label[sample_image])

f, axarr = plt.subplots(1, 2)
axarr[0].imshow(test_set[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % test_label[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()

print(outputs.squeeze())
../_images/sphx_glr_plot_gxnor_mnist_002.png

Out:

Input Label: 7
[-2.4073963 -2.4073963 -3.9689505 -3.6436267 -3.3183029 -3.9038858
 -4.684663   2.4073963 -3.7086916 -3.6436267]

Consider the output from the model, printed above. As is typical in backprop trained models, the final layer here comprises a ‘fully-connected or ‘dense’ layer, with one neuron per class in the data (here, 10). The goal of training is to maximize the response of the neuron corresponding to the label of each training sample, while minimizing the responses of the other neurons.

In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.

Check this for some of the other samples by editing the value of sample_image in the script above (anything from 0 to 9999).

5. Check performance across a number of samples

We’ve included a utility to test performance across a large number of samples. You can run this below.

# Check performance against num_samples samples
num_samples = 10000

results = model_akida.predict(test_set[:int(num_samples)], 10)
accuracy = accuracy_score(test_label[:num_samples], results[:num_samples])
f1 = f1_score(test_label[:num_samples],
              results[:num_samples],
              average='weighted')

# For non-regression purpose
assert accuracy > 0.99

# Print model statistics
print("Model statistics")
stats = model_akida.get_statistics()
model_akida.predict(test_set[:20], 10)
for _, stat in stats.items():
    print(stat)

# Display results
print("Accuracy: " + "{0:.2f}".format(100 * accuracy) + "% / " + "F1 score: " +
      "{0:.2f}".format(f1))

Out:

Model statistics
Layer (type)                  output sparsity
conv_0 (InputConvolutional)   0.85
Layer (type)                  input sparsity      output sparsity     ops
conv_1 (Convolutional)        0.85                0.78                752000
Layer (type)                  input sparsity      output sparsity     ops
conv_2 (Convolutional)        0.78                0.69                270000
Layer (type)                  input sparsity      output sparsity     ops
dense_2 (FullyConnected)      0.69                0.68                246630
Layer (type)                  input sparsity      output sparsity     ops
dense_3 (FullyConnected)      0.68                0.00                1650
Accuracy: 99.07% / F1 score: 0.99

Depending on the number of samples you run, you should find a performance of around 99% (99.35% if you run all 10000 samples).

Note that classification here is done simply by identifying the neuron with the highest activation level. Slightly higher performance is actually possible for this model implementation (~99.1 %) if a very slightly more complex final classification is applied (with a single additional integer subtraction per neuron), but for simplicity we leave those details aside here. See the cnn2snn training framework for a full description.

Total running time of the script: ( 0 minutes 4.838 seconds)

Gallery generated by Sphinx-Gallery