Transfer learning with MobileNet for cats vs. dogs

This tutorial presents a demonstration of transfer learning and the conversion to an Akida model of a quantized Keras network.

The transfer learning example is derived from the Tensorflow tutorial:

  • Our base model is an Akida-compatible version of MobileNet v1, trained on ImageNet.

  • The new dataset for transfer learning is cats vs. dogs (link).

  • We use transfer learning to customize the model to the new task of classifying cats and dogs.

Note

This tutorial only shows the inference of the trained Keras model and its conversion to an Akida network. A textual explanation of the training is given below.

1. Transfer learning process

transfer_learning_image

Transfer learning allows to classify on a specific task by using a pre-trained base model. For an introduction to transfer learning, please refer to the Tensorflow tutorial before exploring this tutorial. Here, we focus on how to quantize the Keras model in order to convert it to an Akida one.

The model is composed of:

  • a base quantized MobileNet model used to extract image features

  • a top layer to classify cats and dogs

  • a sigmoid activation function to interpret model outputs as a probability

Base model

The base model is an Akida-compatible version of MobileNet v1. This model was trained and quantized using the ImageNet dataset. Please refer to the corresponding example for more information. The layers have 4-bit weights (except for the first layer having 8-bit weights) and the activations are quantized to 4 bits. This base model ends with a global average pooling whose output is (1, 1, 1024).

In our transfer learning process, the base model is frozen, i.e., the weights are not updated during training. Pre-trained weights for the quantized model are provided on http://data.brainchip.com/models/mobilenet/. These are loaded in our frozen base model.

Top layer

While the Tensorflow tutorial uses a fully-connected top layer with one output neuron, the only Akida layer supporting 4-bit weights is a separable convolutional layer (see hardware compatibility).

We thus decided to use a separable convolutional layer with one output neuron for the top layer of our model.

Final activation

ReLU6 is the only activation function that can be converted into an Akida SNN equivalent. The converted Akida model doesn’t therefore include the ‘sigmoid’ activation, and we must instead apply it explicitly on the raw values returned by the model Top layer.

Training steps

The transfer learning process consists in two training phases:

  1. Float top layer training: The base model is quantized using 4-bit weights and activations. Pre-trained 4-bit weights of MobileNet/ImageNet are loaded. Then a top layer is added with float weights. The base model is frozen and the training is only applied on the top layer. After 10 epochs, the weights are saved. Note that the weights of the layers of the frozen base model haven’t changed; only those of the top layer are updated.

  2. 4-bit top layer training: The base model is still quantized using 4-bit weights and activations. The added top layer is now quantized (4-bit weights). The weights saved at step 1 are used as initialization. The base model is frozen and the training is only applied on the top layer. After 10 epochs, the new quantized weights are saved. This final weights are those used in the inference below.

Training step

Frozen base model

Init. weights base model

Top layer

Init. weights top layer

E p o c h s

step 1

4-bit weights / activations

pre-trained 4-bit

float weights

random

10

step 2

4-bit weights / activations

pre-trained 4-bit

4-bit weights

saved from step 1

10

import os
import numpy as np
import tensorflow as tf
import tensorflow_datasets as tfds
import matplotlib.pyplot as plt

from akida_models import mobilenet_imagenet
from cnn2snn import convert
from akida_models.quantization_blocks import separable_conv_block

2. Load and preprocess data

In this section, we will load the ‘cats_vs_dogs’ dataset preprocess the data to match the required model’s inputs:

  • 2.A - Load and split data: we only keep the test set which represents 10% of the dataset.

  • 2.B - Preprocess the test set by resizing and rescaling the images.

  • 2.C - Get labels

2.A - Load and split data

The cats_vs_dogs dataset is loaded and split into train, validation and test sets. The train and validation sets were used for the transfer learning process. Here only the test set is used. We use here tf.Dataset objects to load and preprocess batches of data (one can look at the TensorFlow guide here for more information).

Note

The cats_vs_dogs dataset version used here is 4.0.0.

splits = ['train[:80%]', 'train[80%:90%]', 'train[90%:]']

tfds.disable_progress_bar()
(raw_train, raw_validation,
 raw_test), metadata = tfds.load('cats_vs_dogs:4.0.0',
                                 split=splits,
                                 with_info=True,
                                 as_supervised=True)

Out:

Downloading and preparing dataset cats_vs_dogs/4.0.0 (download: 786.68 MiB, generated: Unknown size, total: 786.68 MiB) to /root/tensorflow_datasets/cats_vs_dogs/4.0.0...
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:986: InsecureRequestWarning: Unverified HTTPS request is being made to host 'download.microsoft.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
WARNING:absl:1738 images were corrupted and were skipped
Shuffling and writing examples to /root/tensorflow_datasets/cats_vs_dogs/4.0.0.incompleteU4HECH/cats_vs_dogs-train.tfrecord
Dataset cats_vs_dogs downloaded and prepared to /root/tensorflow_datasets/cats_vs_dogs/4.0.0. Subsequent calls will reuse this data.

2.B - Preprocess the test set

We must apply the same preprocessing as for training: rescaling and resizing. Since Akida models directly accept integer-valued images, we also define a preprocessing function for Akida:

  • for Keras: images are rescaled between 0 and 1, and resized to 160x160

  • for Akida: images are only resized to 160x160 (uint8 values).

Keras and Akida models require 4-dimensional (N,H,W,C) arrays as inputs. We must then create batches of images to feed the model. For inference, the batch size is not relevant; you can set it such that the batch of images can be loaded in memory depending on your CPU/GPU.

IMG_SIZE = 160
input_scaling = (127.5, 127.5)


def format_example_keras(image, label):
    image = tf.cast(image, tf.float32)
    image = (image - input_scaling[1]) / input_scaling[0]
    image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))
    return image, label


def format_example_akida(image, label):
    image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))
    image = tf.cast(image, tf.uint8)
    return image, label
BATCH_SIZE = 32
test_batches_keras = raw_test.map(format_example_keras).batch(BATCH_SIZE)
test_batches_akida = raw_test.map(format_example_akida).batch(BATCH_SIZE)

2.C - Get labels

Labels are contained in the test set as ‘0’ for cats and ‘1’ for dogs. We read through the batches to extract the labels.

labels = np.array([])
for _, label_batch in test_batches_keras:
    labels = np.concatenate((labels, label_batch))

get_label_name = metadata.features['label'].int2str
num_images = labels.shape[0]

print(f"Test set composed of {num_images} images: "
      f"{np.count_nonzero(labels==0)} cats and "
      f"{np.count_nonzero(labels==1)} dogs.")

Out:

Test set composed of 2326 images: 1160 cats and 1166 dogs.

3. Convert a quantized Keras model to Akida

In this section, we will instantiate a quantized Keras model based on MobileNet and modify the last layers to specify the classification for cats_vs_dogs. After loading the pre-trained weights, we will convert the Keras model to Akida.

This section goes as follows:

  • 3.A - Instantiate a Keras base model

  • 3.B - Modify the network and load pre-trained weights

  • 3.C - Convert to Akida

3.A - Instantiate a Keras base model

Here, we instantiate a quantized Keras model based on a MobileNet model. This base model was previously trained using the 1000 classes of the ImageNet dataset. For more information, please see the ImageNet tutorial.

The quantized MobileNet model satisfies the Akida NSoC requirements:

  • The model relies on a convolutional layer (first layer) and separable convolutional layers, all being Akida-compatible.

  • All the separable convolutional layers have 4-bit weights, the first convolutional layer has 8-bit weights.

  • The activations are quantized with 4 bits.

Using the provided quantized MobileNet model, we create an instance without the top classification layer (‘include_top=False’).

base_model_keras = mobilenet_imagenet(input_shape=(IMG_SIZE, IMG_SIZE, 3),
                                      include_top=False,
                                      pooling='avg',
                                      weight_quantization=4,
                                      activ_quantization=4,
                                      input_weight_quantization=8)

Out:

WARNING: Keyword argument 'strides' is not supported in conv_block except for the first layer.

3.B - Modify the network and load pre-trained weights

As explained in section 1, we add a separable convolutional layer as top layer with one output neuron. The new model is now appropriate for the cats_vs_dogs dataset and is Akida-compatible. Note that a sigmoid activation is added at the end of the model: the output neuron returns a probability between 0 and 1 that the input image is a dog.

The transfer learning process has been run internally and the weights have been saved. In this tutorial, the pre-trained weights are loaded for inference and conversion.

Note

The pre-trained weights which are loaded corresponds to the quantization parameters described as above. If you want to modify these parameters, you must re-train the model and save weights.

# Add a top layer for classification
x = base_model_keras.output
x = tf.keras.layers.Reshape((1, 1, 1024), name='reshape_1')(x)
x = separable_conv_block(x,
                         filters=1,
                         kernel_size=(3, 3),
                         padding='same',
                         use_bias=False,
                         name='top_layer_separable',
                         weight_quantization=4,
                         activ_quantization=None)
x = tf.keras.layers.Activation('sigmoid')(x)
preds = tf.keras.layers.Reshape((1,), name='reshape_2')(x)
model_keras = tf.keras.Model(inputs=base_model_keras.input,
                             outputs=preds,
                             name="model_cats_vs_dogs")

model_keras.summary()

Out:

Model: "model_cats_vs_dogs"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_15 (InputLayer)        [(None, 160, 160, 3)]     0
_________________________________________________________________
conv_0 (QuantizedConv2D)     (None, 80, 80, 32)        896
_________________________________________________________________
activation_discrete_relu_14  (None, 80, 80, 32)        0
_________________________________________________________________
separable_1 (QuantizedSepara (None, 80, 80, 64)        2400
_________________________________________________________________
activation_discrete_relu_15  (None, 80, 80, 64)        0
_________________________________________________________________
separable_2 (QuantizedSepara (None, 80, 80, 128)       8896
_________________________________________________________________
separable_2_maxpool (MaxPool (None, 40, 40, 128)       0
_________________________________________________________________
activation_discrete_relu_16  (None, 40, 40, 128)       0
_________________________________________________________________
separable_3 (QuantizedSepara (None, 40, 40, 128)       17664
_________________________________________________________________
activation_discrete_relu_17  (None, 40, 40, 128)       0
_________________________________________________________________
separable_4 (QuantizedSepara (None, 40, 40, 256)       34176
_________________________________________________________________
separable_4_maxpool (MaxPool (None, 20, 20, 256)       0
_________________________________________________________________
activation_discrete_relu_18  (None, 20, 20, 256)       0
_________________________________________________________________
separable_5 (QuantizedSepara (None, 20, 20, 256)       68096
_________________________________________________________________
activation_discrete_relu_19  (None, 20, 20, 256)       0
_________________________________________________________________
separable_6 (QuantizedSepara (None, 20, 20, 512)       133888
_________________________________________________________________
separable_6_maxpool (MaxPool (None, 10, 10, 512)       0
_________________________________________________________________
activation_discrete_relu_20  (None, 10, 10, 512)       0
_________________________________________________________________
separable_7 (QuantizedSepara (None, 10, 10, 512)       267264
_________________________________________________________________
activation_discrete_relu_21  (None, 10, 10, 512)       0
_________________________________________________________________
separable_8 (QuantizedSepara (None, 10, 10, 512)       267264
_________________________________________________________________
activation_discrete_relu_22  (None, 10, 10, 512)       0
_________________________________________________________________
separable_9 (QuantizedSepara (None, 10, 10, 512)       267264
_________________________________________________________________
activation_discrete_relu_23  (None, 10, 10, 512)       0
_________________________________________________________________
separable_10 (QuantizedSepar (None, 10, 10, 512)       267264
_________________________________________________________________
activation_discrete_relu_24  (None, 10, 10, 512)       0
_________________________________________________________________
separable_11 (QuantizedSepar (None, 10, 10, 512)       267264
_________________________________________________________________
activation_discrete_relu_25  (None, 10, 10, 512)       0
_________________________________________________________________
separable_12 (QuantizedSepar (None, 10, 10, 1024)      529920
_________________________________________________________________
separable_12_maxpool (MaxPoo (None, 5, 5, 1024)        0
_________________________________________________________________
activation_discrete_relu_26  (None, 5, 5, 1024)        0
_________________________________________________________________
separable_13 (QuantizedSepar (None, 5, 5, 1024)        1058816
_________________________________________________________________
separable_13_global_avg (Glo (None, 1024)              0
_________________________________________________________________
activation_discrete_relu_27  (None, 1024)              0
_________________________________________________________________
reshape_1 (Reshape)          (None, 1, 1, 1024)        0
_________________________________________________________________
top_layer_separable (Quantiz (None, 1, 1, 1)           10240
_________________________________________________________________
activation (Activation)      (None, 1, 1, 1)           0
_________________________________________________________________
reshape_2 (Reshape)          (None, 1)                 0
=================================================================
Total params: 3,201,312
Trainable params: 3,201,312
Non-trainable params: 0
_________________________________________________________________
from cnn2snn import load_quantized_model

# Load pre-trained model
pretrained_model = tf.keras.utils.get_file(
    "mobilenet_cats_vs_dogs_iq8_wq4_aq4.h5",
    "http://data.brainchip.com/models/mobilenet/mobilenet_cats_vs_dogs_iq8_wq4_aq4.h5",
    cache_subdir='models/mobilenet')
model_keras = load_quantized_model(pretrained_model)

Out:

Downloading data from http://data.brainchip.com/models/mobilenet/mobilenet_cats_vs_dogs_iq8_wq4_aq4.h5

    8192/13030056 [..............................] - ETA: 55s
   73728/13030056 [..............................] - ETA: 16s
  270336/13030056 [..............................] - ETA: 7s 
  466944/13030056 [>.............................] - ETA: 5s
  663552/13030056 [>.............................] - ETA: 5s
  860160/13030056 [>.............................] - ETA: 4s
 1056768/13030056 [=>............................] - ETA: 4s
 1253376/13030056 [=>............................] - ETA: 4s
 1449984/13030056 [==>...........................] - ETA: 4s
 1646592/13030056 [==>...........................] - ETA: 3s
 1843200/13030056 [===>..........................] - ETA: 3s
 2039808/13030056 [===>..........................] - ETA: 3s
 2236416/13030056 [====>.........................] - ETA: 3s
 2433024/13030056 [====>.........................] - ETA: 3s
 2629632/13030056 [=====>........................] - ETA: 3s
 2826240/13030056 [=====>........................] - ETA: 3s
 3022848/13030056 [=====>........................] - ETA: 3s
 3219456/13030056 [======>.......................] - ETA: 3s
 3416064/13030056 [======>.......................] - ETA: 3s
 3612672/13030056 [=======>......................] - ETA: 3s
 3809280/13030056 [=======>......................] - ETA: 2s
 4005888/13030056 [========>.....................] - ETA: 2s
 4202496/13030056 [========>.....................] - ETA: 2s
 4399104/13030056 [=========>....................] - ETA: 2s
 4595712/13030056 [=========>....................] - ETA: 2s
 4792320/13030056 [==========>...................] - ETA: 2s
 4988928/13030056 [==========>...................] - ETA: 2s
 5185536/13030056 [==========>...................] - ETA: 2s
 5382144/13030056 [===========>..................] - ETA: 2s
 5578752/13030056 [===========>..................] - ETA: 2s
 5775360/13030056 [============>.................] - ETA: 2s
 5971968/13030056 [============>.................] - ETA: 2s
 6168576/13030056 [=============>................] - ETA: 2s
 6365184/13030056 [=============>................] - ETA: 2s
 6561792/13030056 [==============>...............] - ETA: 2s
 6758400/13030056 [==============>...............] - ETA: 1s
 6955008/13030056 [===============>..............] - ETA: 1s
 7151616/13030056 [===============>..............] - ETA: 1s
 7348224/13030056 [===============>..............] - ETA: 1s
 7544832/13030056 [================>.............] - ETA: 1s
 7741440/13030056 [================>.............] - ETA: 1s
 7938048/13030056 [=================>............] - ETA: 1s
 8134656/13030056 [=================>............] - ETA: 1s
 8331264/13030056 [==================>...........] - ETA: 1s
 8527872/13030056 [==================>...........] - ETA: 1s
 8724480/13030056 [===================>..........] - ETA: 1s
 8921088/13030056 [===================>..........] - ETA: 1s
 9117696/13030056 [===================>..........] - ETA: 1s
 9314304/13030056 [====================>.........] - ETA: 1s
 9510912/13030056 [====================>.........] - ETA: 1s
 9707520/13030056 [=====================>........] - ETA: 1s
 9904128/13030056 [=====================>........] - ETA: 0s
10100736/13030056 [======================>.......] - ETA: 0s
10297344/13030056 [======================>.......] - ETA: 0s
10493952/13030056 [=======================>......] - ETA: 0s
10690560/13030056 [=======================>......] - ETA: 0s
10887168/13030056 [========================>.....] - ETA: 0s
11083776/13030056 [========================>.....] - ETA: 0s
11280384/13030056 [========================>.....] - ETA: 0s
11476992/13030056 [=========================>....] - ETA: 0s
11673600/13030056 [=========================>....] - ETA: 0s
11870208/13030056 [==========================>...] - ETA: 0s
12066816/13030056 [==========================>...] - ETA: 0s
12263424/13030056 [===========================>..] - ETA: 0s
12460032/13030056 [===========================>..] - ETA: 0s
12656640/13030056 [============================>.] - ETA: 0s
12853248/13030056 [============================>.] - ETA: 0s
13033472/13030056 [==============================] - 4s 0us/step

3.C - Convert to Akida

The new Keras model with pre-trained weights is now converted to an Akida model. It only requires the quantized Keras model and the inputs scaling used during training. Note: the ‘sigmoid’ activation has no SNN equivalent and will be simply ignored during the conversion.

model_akida = convert(model_keras, input_scaling=input_scaling)

model_akida.summary()

Out:

Warning: the activation layer 'activation' will be discarded at conversion. The outputs of the Akida model will be the potentials before this activation layer.
                                          Model Summary
__________________________________________________________________________________________________
Layer (type)                                  Output shape   Kernel shape
==================================================================================================
conv_0 (InputConvolutional)                   [80, 80, 32]   (3, 3, 3, 32)
__________________________________________________________________________________________________
separable_1 (SeparableConvolutional)          [80, 80, 64]   (3, 3, 32, 1), (1, 1, 32, 64)
__________________________________________________________________________________________________
separable_2 (SeparableConvolutional)          [40, 40, 128]  (3, 3, 64, 1), (1, 1, 64, 128)
__________________________________________________________________________________________________
separable_3 (SeparableConvolutional)          [40, 40, 128]  (3, 3, 128, 1), (1, 1, 128, 128)
__________________________________________________________________________________________________
separable_4 (SeparableConvolutional)          [20, 20, 256]  (3, 3, 128, 1), (1, 1, 128, 256)
__________________________________________________________________________________________________
separable_5 (SeparableConvolutional)          [20, 20, 256]  (3, 3, 256, 1), (1, 1, 256, 256)
__________________________________________________________________________________________________
separable_6 (SeparableConvolutional)          [10, 10, 512]  (3, 3, 256, 1), (1, 1, 256, 512)
__________________________________________________________________________________________________
separable_7 (SeparableConvolutional)          [10, 10, 512]  (3, 3, 512, 1), (1, 1, 512, 512)
__________________________________________________________________________________________________
separable_8 (SeparableConvolutional)          [10, 10, 512]  (3, 3, 512, 1), (1, 1, 512, 512)
__________________________________________________________________________________________________
separable_9 (SeparableConvolutional)          [10, 10, 512]  (3, 3, 512, 1), (1, 1, 512, 512)
__________________________________________________________________________________________________
separable_10 (SeparableConvolutional)         [10, 10, 512]  (3, 3, 512, 1), (1, 1, 512, 512)
__________________________________________________________________________________________________
separable_11 (SeparableConvolutional)         [10, 10, 512]  (3, 3, 512, 1), (1, 1, 512, 512)
__________________________________________________________________________________________________
separable_12 (SeparableConvolutional)         [5, 5, 1024]   (3, 3, 512, 1), (1, 1, 512, 1024)
__________________________________________________________________________________________________
separable_13 (SeparableConvolutional)         [1, 1, 1024]   (3, 3, 1024, 1), (1, 1, 1024, 1024)
__________________________________________________________________________________________________
top_layer_separable (SeparableConvolutional)  [1, 1, 1]      (3, 3, 1024, 1), (1, 1, 1024, 1)
__________________________________________________________________________________________________
Input shape: 160, 160, 3
Backend type: Software - 1.8.8

4. Classify test images

This section gives a comparison of the results between the quantized Keras and the Akida models. It goes as follows:

  • 4.A - Classify test images with the quantized Keras and the Akida models

  • 4.B - Compare results

4.A Classify test images

Here, we will predict the classes of the test images using the quantized Keras model and the converted Akida model. Remember that:

  • Input images in Keras and Akida are not scaled in the same range, be careful to use the correct inputs: uint8 images for Akida and float rescaled images for Keras.

  • The predict function of tf.keras can take a tf.data.Dataset object as argument. However, the Akida evaluate function takes a NumPy array containing the images. Though the Akida predict function exists, it outputs a class label and not the raw predictions.

  • The Keras predict function returns the probability to be a dog: if the output is greater than 0.5, the model predicts a ‘dog’. However, the Akida evaluate function directly returns the potential before the ‘sigmoid’ activation, which has no SNN equivalent. We must therefore apply it explicitly on the model outputs to obtain the Akida probabilities.

# Classify test images with the quantized Keras model
from timeit import default_timer as timer

start = timer()
pots_keras = model_keras.predict(test_batches_keras)
end = timer()

preds_keras = pots_keras.squeeze() > 0.5
print(f"Keras inference on {num_images} images took {end-start:.2f} s.\n")

Out:

Keras inference on 2326 images took 9.94 s.
# Classify test images with the Akida model
from progressbar import ProgressBar
n_batches = num_images // BATCH_SIZE + 1
pbar = ProgressBar(maxval=n_batches)
i = 1
pbar.start()
start = timer()
pots_akida = np.array([], dtype=np.float32)
for batch, _ in test_batches_akida:
    pots_batch_akida = model_akida.evaluate(batch.numpy())
    pots_akida = np.concatenate((pots_akida, pots_batch_akida.squeeze()))
    pbar.update(i)
    i = i + 1
pbar.finish()
end = timer()

preds_akida = tf.keras.layers.Activation('sigmoid')(pots_akida) > 0.5
print(f"Akida inference on {num_images} images took {end-start:.2f} s.\n")

Out:

  0% |                                                                        |
  1% |                                                                        |
  2% |#                                                                       |
  4% |##                                                                      |
  5% |###                                                                     |
  6% |####                                                                    |
  8% |#####                                                                   |
  9% |######                                                                  |
 10% |#######                                                                 |
 12% |########                                                                |
 13% |#########                                                               |
 15% |##########                                                              |
 16% |###########                                                             |
 17% |############                                                            |
 19% |#############                                                           |
 20% |##############                                                          |
 21% |###############                                                         |
 23% |################                                                        |
 24% |#################                                                       |
 26% |##################                                                      |
 27% |###################                                                     |
 28% |####################                                                    |
 30% |#####################                                                   |
 31% |######################                                                  |
 32% |#######################                                                 |
 34% |########################                                                |
 35% |#########################                                               |
 36% |##########################                                              |
 38% |###########################                                             |
 39% |############################                                            |
 41% |#############################                                           |
 42% |##############################                                          |
 43% |###############################                                         |
 45% |################################                                        |
 46% |#################################                                       |
 47% |##################################                                      |
 49% |###################################                                     |
 50% |####################################                                    |
 52% |#####################################                                   |
 53% |######################################                                  |
 54% |#######################################                                 |
 56% |########################################                                |
 57% |#########################################                               |
 58% |##########################################                              |
 60% |###########################################                             |
 61% |############################################                            |
 63% |#############################################                           |
 64% |##############################################                          |
 65% |###############################################                         |
 67% |################################################                        |
 68% |#################################################                       |
 69% |##################################################                      |
 71% |###################################################                     |
 72% |####################################################                    |
 73% |#####################################################                   |
 75% |######################################################                  |
 76% |#######################################################                 |
 78% |########################################################                |
 79% |#########################################################               |
 80% |##########################################################              |
 82% |###########################################################             |
 83% |############################################################            |
 84% |#############################################################           |
 86% |##############################################################          |
 87% |###############################################################         |
 89% |################################################################        |
 90% |#################################################################       |
 91% |##################################################################      |
 93% |###################################################################     |
 94% |####################################################################    |
 95% |#####################################################################   |
 97% |######################################################################  |
 98% |####################################################################### |
100% |########################################################################|
100% |########################################################################|
Akida inference on 2326 images took 42.81 s.
# Print model statistics
print("Model statistics")
stats = model_akida.get_statistics()
batch, _ = iter(test_batches_akida).get_next()
model_akida.evaluate(batch[:20].numpy())
for _, stat in stats.items():
    print(stat)

Out:

Model statistics
Layer (type)                  output sparsity
conv_0 (InputConvolutional)   0.31
Layer (type)                  input sparsity      output sparsity     ops
separable_1 (SeparableConvolu 0.31                0.34                82084362
Layer (type)                  input sparsity      output sparsity     ops
separable_2 (SeparableConvolu 0.34                0.32                315577386
Layer (type)                  input sparsity      output sparsity     ops
separable_3 (SeparableConvolu 0.32                0.33                160853648
Layer (type)                  input sparsity      output sparsity     ops
separable_4 (SeparableConvolu 0.33                0.47                316108110
Layer (type)                  input sparsity      output sparsity     ops
separable_5 (SeparableConvolu 0.47                0.36                124863488
Layer (type)                  input sparsity      output sparsity     ops
separable_6 (SeparableConvolu 0.36                0.54                302764163
Layer (type)                  input sparsity      output sparsity     ops
separable_7 (SeparableConvolu 0.54                0.58                108423550
Layer (type)                  input sparsity      output sparsity     ops
separable_8 (SeparableConvolu 0.58                0.63                99812845
Layer (type)                  input sparsity      output sparsity     ops
separable_9 (SeparableConvolu 0.63                0.71                86568982
Layer (type)                  input sparsity      output sparsity     ops
separable_10 (SeparableConvol 0.71                0.69                69379891
Layer (type)                  input sparsity      output sparsity     ops
separable_11 (SeparableConvol 0.69                0.66                73416302
Layer (type)                  input sparsity      output sparsity     ops
separable_12 (SeparableConvol 0.66                0.85                161363700
Layer (type)                  input sparsity      output sparsity     ops
separable_13 (SeparableConvol 0.85                0.57                35251494
Layer (type)                  input sparsity      output sparsity     ops
top_layer_separable (Separabl 0.57                0.00                7852

4.B Compare results

The Keras and Akida accuracies are compared and the Akida confusion matrix is given (the quantized Keras confusion matrix is almost identical to the Akida one). Note that there is no exact equivalence between the quantized Keras and the Akida models. However, the accuracies are highly similar.

# Compute accuracies
n_good_preds_keras = np.sum(np.equal(preds_keras, labels))
n_good_preds_akida = np.sum(np.equal(preds_akida, labels))

keras_accuracy = n_good_preds_keras / num_images
akida_accuracy = n_good_preds_akida / num_images

print(f"Quantized Keras accuracy: {keras_accuracy*100:.2f} %  "
      f"({n_good_preds_keras} / {num_images} images)")
print(f"Akida accuracy:           {akida_accuracy*100:.2f} %  "
      f"({n_good_preds_akida} / {num_images} images)")

# For non-regression purpose
assert akida_accuracy > 0.97

Out:

Quantized Keras accuracy: 97.59 %  (2270 / 2326 images)
Akida accuracy:           97.46 %  (2267 / 2326 images)
def confusion_matrix_2classes(labels, predictions):
    tp = np.count_nonzero(labels + predictions == 2)
    tn = np.count_nonzero(labels + predictions == 0)
    fp = np.count_nonzero(predictions - labels == 1)
    fn = np.count_nonzero(labels - predictions == 1)

    return np.array([[tp, fn], [fp, tn]])


def plot_confusion_matrix_2classes(cm, classes):
    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]

    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)
    plt.xticks([0, 1], classes)
    plt.yticks([0, 1], classes)

    for i, j in zip([0, 0, 1, 1], [0, 1, 0, 1]):
        plt.text(j,
                 i,
                 f"{cm[i, j]:.2f}",
                 horizontalalignment="center",
                 color="white" if cm[i, j] > cm.max() / 2. else "black")

    plt.ylabel('True label')
    plt.xlabel('Predicted label')
    plt.autoscale()
# Plot confusion matrix for Akida
cm_akida = confusion_matrix_2classes(labels, preds_akida.numpy())
print("Confusion matrix quantized Akida:")
plot_confusion_matrix_2classes(cm_akida, ['dog', 'cat'])
plt.show()
../_images/sphx_glr_plot_transfer_learning_001.png

Out:

Confusion matrix quantized Akida:

Total running time of the script: ( 1 minutes 44.606 seconds)

Gallery generated by Sphinx-Gallery