GXNOR/MNIST inference

The MNIST dataset is a handwritten digits database. It has a training set of 60,000 samples, and a test set of 10,000 samples. Each sample comprises a 28x28 pixel image and an associated label.

This tutorial illustrates how to use a pre-trained model to process the MNIST dataset.

1. Dataset preparation

import numpy as np

import matplotlib.cm as cm
import matplotlib.pyplot as plt

from tensorflow.keras.datasets import mnist

# Retrieve MNIST dataset
_, (test_set, test_label) = mnist.load_data()

# Add a dimension to images sets as akida expects 4 dimensions inputs
test_set = np.expand_dims(test_set, -1)

# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
    axarr[i].imshow(test_set[i].reshape((28, 28)), cmap=cm.Greys_r)
    axarr[i].set_title('Class %d' % test_label[i])
plt.show()
Class 7, Class 2, Class 1, Class 0

Out:

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz

   16384/11490434 [..............................] - ETA: 0s
  376832/11490434 [..............................] - ETA: 1s
 1441792/11490434 [==>...........................] - ETA: 0s
 2490368/11490434 [=====>........................] - ETA: 0s
 3817472/11490434 [========>.....................] - ETA: 0s
 4907008/11490434 [===========>..................] - ETA: 0s
 6021120/11490434 [==============>...............] - ETA: 0s
 7135232/11490434 [=================>............] - ETA: 0s
 8249344/11490434 [====================>.........] - ETA: 0s
 8921088/11490434 [======================>.......] - ETA: 0s
 9707520/11490434 [========================>.....] - ETA: 0s
11223040/11490434 [============================>.] - ETA: 0s
11493376/11490434 [==============================] - 1s 0us/step

11501568/11490434 [==============================] - 1s 0us/step

2. Create a Keras GXNOR model

The GXNOR architecture is available in the Akida models zoo along with pretrained weights.

Note

The pre-trained weights were obtained with knowledge distillation training, using the EfficientNet model from this repository and the Distiller class from the knowledge distillation toolkit.

The float training was done for 30 epochs, the model is then gradually quantized following: 8-4-4 –> 4-4-4 –> 4-4-2 –> 2-2-2 –> 2-2-1 by tuning the model at each step with the same distillation training method for 5 epochs.

from akida_models import gxnor_mnist_pretrained

model_keras = gxnor_mnist_pretrained()
model_keras.summary()

Out:

Downloading data from http://data.brainchip.com/models/gxnor/gxnor_mnist_iq2_wq2_aq1.h5

  16384/6684600 [..............................] - ETA: 5s
 139264/6684600 [..............................] - ETA: 3s
 401408/6684600 [>.............................] - ETA: 1s
 663552/6684600 [=>............................] - ETA: 1s
 925696/6684600 [===>..........................] - ETA: 1s
1187840/6684600 [====>.........................] - ETA: 1s
1449984/6684600 [=====>........................] - ETA: 1s
1712128/6684600 [======>.......................] - ETA: 1s
1974272/6684600 [=======>......................] - ETA: 1s
2236416/6684600 [=========>....................] - ETA: 1s
2498560/6684600 [==========>...................] - ETA: 0s
2760704/6684600 [===========>..................] - ETA: 0s
3022848/6684600 [============>.................] - ETA: 0s
3284992/6684600 [=============>................] - ETA: 0s
3547136/6684600 [==============>...............] - ETA: 0s
3809280/6684600 [================>.............] - ETA: 0s
4071424/6684600 [=================>............] - ETA: 0s
4333568/6684600 [==================>...........] - ETA: 0s
4595712/6684600 [===================>..........] - ETA: 0s
4857856/6684600 [====================>.........] - ETA: 0s
5120000/6684600 [=====================>........] - ETA: 0s
5382144/6684600 [=======================>......] - ETA: 0s
5644288/6684600 [========================>.....] - ETA: 0s
5906432/6684600 [=========================>....] - ETA: 0s
6168576/6684600 [==========================>...] - ETA: 0s
6430720/6684600 [===========================>..] - ETA: 0s
6684672/6684600 [==============================] - 2s 0us/step

6692864/6684600 [==============================] - 2s 0us/step
WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
Model: "gxnor_mnist"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 28, 28, 1)]       0
_________________________________________________________________
rescaling (Rescaling)        (None, 28, 28, 1)         0
_________________________________________________________________
block1_conv1 (QuantizedConv2 (None, 28, 28, 32)        832
_________________________________________________________________
block1_conv1_maxpool (MaxPoo (None, 14, 14, 32)        0
_________________________________________________________________
block1_conv1_relu (Activatio (None, 14, 14, 32)        0
_________________________________________________________________
block2_conv1 (QuantizedConv2 (None, 14, 14, 64)        51264
_________________________________________________________________
block2_conv1_maxpool (MaxPoo (None, 7, 7, 64)          0
_________________________________________________________________
block2_conv1_relu (Activatio (None, 7, 7, 64)          0
_________________________________________________________________
flatten (Flatten)            (None, 3136)              0
_________________________________________________________________
fc1 (QuantizedDense)         (None, 512)               1606144
_________________________________________________________________
fc1_relu (ActivationDiscrete (None, 512)               0
_________________________________________________________________
predictions (QuantizedDense) (None, 10)                5130
=================================================================
Total params: 1,663,370
Trainable params: 1,663,370
Non-trainable params: 0
_________________________________________________________________
# Check Model performances
from tensorflow.keras.losses import SparseCategoricalCrossentropy

model_keras.compile(optimizer='adam',
                    loss=SparseCategoricalCrossentropy(from_logits=True),
                    metrics=['accuracy'])
keras_accuracy = model_keras.evaluate(test_set, test_label, verbose=0)[1]
print(f"Keras accuracy : {keras_accuracy}")

Out:

Keras accuracy : 0.9923999905586243

3. Conversion to Akida

3.1 Convert to Akida model

When converting to an Akida model, we just need to pass the Keras model to cnn2snn.convert.

from cnn2snn import convert

model_akida = convert(model_keras)

3.2 Check hardware compliancy

The Model.summary method provides a detailed description of the Model layers.

model_akida.summary()

Out:

                Model Summary
______________________________________________
Input shape  Output shape  Sequences  Layers
==============================================
[28, 28, 1]  [1, 1, 10]    1          4
______________________________________________

           SW/block1_conv1-predictions (Software)
____________________________________________________________
Layer (type)               Output shape  Kernel shape
============================================================
block1_conv1 (InputConv.)  [14, 14, 32]  (5, 5, 1, 32)
____________________________________________________________
block2_conv1 (Conv.)       [7, 7, 64]    (5, 5, 32, 64)
____________________________________________________________
fc1 (Fully.)               [1, 1, 512]   (1, 1, 3136, 512)
____________________________________________________________
predictions (Fully.)       [1, 1, 10]    (1, 1, 512, 10)
____________________________________________________________

3.3. Check performance

from sklearn.metrics import accuracy_score

# Check performance against num_samples samples
num_samples = 10000

results = model_akida.predict(test_set[:num_samples])
accuracy = accuracy_score(test_label[:num_samples], results[:num_samples])

# For non-regression purpose
assert accuracy > 0.99

# Display results
print("Accuracy: " + "{0:.2f}".format(100 * accuracy) + "%")

Out:

Accuracy: 99.24%

Depending on the number of samples you run, you should find a performance of around 99% (99.24% if you run all 10000 samples).

3.4 Show predictions for a single image

Now try processing a single image, say, the first image in the dataset that we looked at above:

# Test a single example
sample_image = 0
image = test_set[sample_image]
outputs = model_akida.evaluate(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % test_label[sample_image])

f, axarr = plt.subplots(1, 2)
axarr[0].imshow(test_set[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % test_label[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()

print(outputs.squeeze())
Class 7

Out:

Input Label: 7
[-2.6885889 -3.3714404 -3.4437032 -2.9686317 -3.1043468 -2.8324056
 -3.2131     3.6556103 -3.326526  -2.5987935]

Consider the output from the model, printed above. As is typical in backprop trained models, the final layer here comprises a ‘fully-connected or ‘dense’ layer, with one neuron per class in the data (here, 10). The goal of training is to maximize the response of the neuron corresponding to the label of each training sample, while minimizing the responses of the other neurons.

In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.

Check this for some of the other samples by editing the value of sample_image in the script above (anything from 0 to 9999).

Total running time of the script: ( 0 minutes 9.706 seconds)

Gallery generated by Sphinx-Gallery