VGG and DS-CNN/CIFAR10 inference

The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

This tutorial uses this dataset for a classic object classification task (airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck).

We start from state-of-the-art CNN models, illustrating how these models can be quantized, then converted to an Akida model with an equivalent accuracy.

The neural networks used in this tutorial are inspired from the VGG and MobileNets architecture respectively.

The VGG architecture uses Convolutional and Dense layers: these layers must therefore be quantized with at most 2 bits of precision to be compatible with the Akida NSoC. This causes a 2 % drop in accuracy.

The DS-CNN architecture uses Separable Convolutional layers that can be quantized using 4 bits of precision, allowing to preserve the base Keras model accuracy.

Model

Accuracy

VGG Keras

93.23 %

VGG Keras quantized

90.59 %

VGG Akida

90.67 %

DS-CNN Keras

93.49 %

DS-CNN Keras quantized

93.07 %

DS-CNN Akida

93.22 %

1. Load CNN2SNN tool dependencies

# System imports
import os
import sys
import numpy as np
from sklearn.metrics import accuracy_score
from timeit import default_timer as timer

# TensorFlow imports
from tensorflow.keras.datasets import cifar10

# Akida models imports
from akida_models import ds_cnn_cifar10_pretrained, vgg_cifar10_pretrained

# CNN2SNN
from cnn2snn import convert

2. Load and reshape CIFAR10 dataset

# Load CIFAR10 dataset
(x_train, y_train), (x_test, y_test) = cifar10.load_data()

# Reshape x-data
x_train = x_train.reshape(50000, 32, 32, 3)
x_test = x_test.reshape(10000, 32, 32, 3)
input_shape = (32, 32, 3)

# Set aside raw test data for use with Akida Execution Engine later
raw_x_test = x_test.astype('uint8')

# Rescale x-data
a = 255
b = 0
input_scaling = (a, b)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train = (x_train - b) / a
x_test = (x_test - b) / a

Out:

Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz

     8192/170498071 [..............................] - ETA: 33:57
    40960/170498071 [..............................] - ETA: 13:35
    90112/170498071 [..............................] - ETA: 9:17 
   204800/170498071 [..............................] - ETA: 5:26
   417792/170498071 [..............................] - ETA: 3:20
   860160/170498071 [..............................] - ETA: 1:56
  1368064/170498071 [..............................] - ETA: 1:44
  2383872/170498071 [..............................] - ETA: 1:02
  4038656/170498071 [..............................] - ETA: 40s 
  4349952/170498071 [..............................] - ETA: 41s
  5955584/170498071 [>.............................] - ETA: 32s
  6627328/170498071 [>.............................] - ETA: 30s
  6955008/170498071 [>.............................] - ETA: 30s
  7479296/170498071 [>.............................] - ETA: 30s
  8167424/170498071 [>.............................] - ETA: 28s
  8544256/170498071 [>.............................] - ETA: 28s
  9035776/170498071 [>.............................] - ETA: 28s
  9723904/170498071 [>.............................] - ETA: 26s
 10051584/170498071 [>.............................] - ETA: 26s
 10559488/170498071 [>.............................] - ETA: 26s
 10919936/170498071 [>.............................] - ETA: 26s
 11411456/170498071 [=>............................] - ETA: 26s
 12115968/170498071 [=>............................] - ETA: 25s
 12476416/170498071 [=>............................] - ETA: 25s
 12992512/170498071 [=>............................] - ETA: 24s
 13344768/170498071 [=>............................] - ETA: 24s
 13836288/170498071 [=>............................] - ETA: 24s
 14573568/170498071 [=>............................] - ETA: 23s
 14934016/170498071 [=>............................] - ETA: 23s
 15474688/170498071 [=>............................] - ETA: 23s
 15818752/170498071 [=>............................] - ETA: 23s
 16326656/170498071 [=>............................] - ETA: 23s
 17063936/170498071 [==>...........................] - ETA: 22s
 17358848/170498071 [==>...........................] - ETA: 22s
 17948672/170498071 [==>...........................] - ETA: 22s
 18292736/170498071 [==>...........................] - ETA: 22s
 18849792/170498071 [==>...........................] - ETA: 22s
 19603456/170498071 [==>...........................] - ETA: 21s
 19849216/170498071 [==>...........................] - ETA: 21s
 20488192/170498071 [==>...........................] - ETA: 21s
 20799488/170498071 [==>...........................] - ETA: 21s
 21405696/170498071 [==>...........................] - ETA: 21s
 22175744/170498071 [==>...........................] - ETA: 20s
 22437888/170498071 [==>...........................] - ETA: 20s
 23093248/170498071 [===>..........................] - ETA: 20s
 23404544/170498071 [===>..........................] - ETA: 20s
 24010752/170498071 [===>..........................] - ETA: 20s
 24780800/170498071 [===>..........................] - ETA: 20s
 25010176/170498071 [===>..........................] - ETA: 20s
 25698304/170498071 [===>..........................] - ETA: 19s
 25976832/170498071 [===>..........................] - ETA: 19s
 26648576/170498071 [===>..........................] - ETA: 19s
 27418624/170498071 [===>..........................] - ETA: 19s
 27631616/170498071 [===>..........................] - ETA: 19s
 28352512/170498071 [===>..........................] - ETA: 19s
 28647424/170498071 [====>.........................] - ETA: 19s
 29302784/170498071 [====>.........................] - ETA: 18s
 30089216/170498071 [====>.........................] - ETA: 18s
 30269440/170498071 [====>.........................] - ETA: 18s
 31023104/170498071 [====>.........................] - ETA: 18s
 31309824/170498071 [====>.........................] - ETA: 18s
 31989760/170498071 [====>.........................] - ETA: 18s
 32776192/170498071 [====>.........................] - ETA: 18s
 32956416/170498071 [====>.........................] - ETA: 18s
 33677312/170498071 [====>.........................] - ETA: 17s
 33906688/170498071 [====>.........................] - ETA: 17s
 34627584/170498071 [=====>........................] - ETA: 17s
 34922496/170498071 [=====>........................] - ETA: 17s
 35594240/170498071 [=====>........................] - ETA: 17s
 36380672/170498071 [=====>........................] - ETA: 17s
 36560896/170498071 [=====>........................] - ETA: 17s
 37298176/170498071 [=====>........................] - ETA: 17s
 37527552/170498071 [=====>........................] - ETA: 17s
 38248448/170498071 [=====>........................] - ETA: 16s
 38494208/170498071 [=====>........................] - ETA: 16s
 39215104/170498071 [=====>........................] - ETA: 16s
 39477248/170498071 [=====>........................] - ETA: 16s
 40148992/170498071 [======>.......................] - ETA: 16s
 40919040/170498071 [======>.......................] - ETA: 16s
 41099264/170498071 [======>.......................] - ETA: 16s
 41869312/170498071 [======>.......................] - ETA: 16s
 42098688/170498071 [======>.......................] - ETA: 16s
 42835968/170498071 [======>.......................] - ETA: 16s
 43048960/170498071 [======>.......................] - ETA: 16s
 43802624/170498071 [======>.......................] - ETA: 15s
 44572672/170498071 [======>.......................] - ETA: 15s
 44769280/170498071 [======>.......................] - ETA: 15s
 45539328/170498071 [=======>......................] - ETA: 15s
 45752320/170498071 [=======>......................] - ETA: 15s
 46505984/170498071 [=======>......................] - ETA: 15s
 46718976/170498071 [=======>......................] - ETA: 15s
 47456256/170498071 [=======>......................] - ETA: 15s
 47685632/170498071 [=======>......................] - ETA: 15s
 48406528/170498071 [=======>......................] - ETA: 15s
 49176576/170498071 [=======>......................] - ETA: 14s
 49373184/170498071 [=======>......................] - ETA: 14s
 50143232/170498071 [=======>......................] - ETA: 14s
 50388992/170498071 [=======>......................] - ETA: 14s
 51126272/170498071 [=======>......................] - ETA: 14s
 51355648/170498071 [========>.....................] - ETA: 14s
 52092928/170498071 [========>.....................] - ETA: 14s
 52862976/170498071 [========>.....................] - ETA: 14s
 53059584/170498071 [========>.....................] - ETA: 14s
 53813248/170498071 [========>.....................] - ETA: 14s
 54075392/170498071 [========>.....................] - ETA: 14s
 54812672/170498071 [========>.....................] - ETA: 14s
 55042048/170498071 [========>.....................] - ETA: 14s
 55762944/170498071 [========>.....................] - ETA: 13s
 56532992/170498071 [========>.....................] - ETA: 13s
 56745984/170498071 [========>.....................] - ETA: 13s
 57499648/170498071 [=========>....................] - ETA: 13s
 57761792/170498071 [=========>....................] - ETA: 13s
 58482688/170498071 [=========>....................] - ETA: 13s
 58728448/170498071 [=========>....................] - ETA: 13s
 59449344/170498071 [=========>....................] - ETA: 13s
 60219392/170498071 [=========>....................] - ETA: 13s
 60432384/170498071 [=========>....................] - ETA: 13s
 61186048/170498071 [=========>....................] - ETA: 13s
 61415424/170498071 [=========>....................] - ETA: 13s
 62152704/170498071 [=========>....................] - ETA: 12s
 62414848/170498071 [=========>....................] - ETA: 12s
 63135744/170498071 [==========>...................] - ETA: 12s
 63905792/170498071 [==========>...................] - ETA: 12s
 64118784/170498071 [==========>...................] - ETA: 12s
 64888832/170498071 [==========>...................] - ETA: 12s
 65134592/170498071 [==========>...................] - ETA: 12s
 65855488/170498071 [==========>...................] - ETA: 12s
 66101248/170498071 [==========>...................] - ETA: 12s
 66805760/170498071 [==========>...................] - ETA: 12s
 67575808/170498071 [==========>...................] - ETA: 12s
 67821568/170498071 [==========>...................] - ETA: 12s
 68575232/170498071 [===========>..................] - ETA: 12s
 68771840/170498071 [===========>..................] - ETA: 12s
 69509120/170498071 [===========>..................] - ETA: 11s
 69754880/170498071 [===========>..................] - ETA: 11s
 70475776/170498071 [===========>..................] - ETA: 11s
 70705152/170498071 [===========>..................] - ETA: 11s
 71409664/170498071 [===========>..................] - ETA: 11s
 72196096/170498071 [===========>..................] - ETA: 11s
 72441856/170498071 [===========>..................] - ETA: 11s
 73195520/170498071 [===========>..................] - ETA: 11s
 73392128/170498071 [===========>..................] - ETA: 11s
 74129408/170498071 [============>.................] - ETA: 11s
 74358784/170498071 [============>.................] - ETA: 11s
 75079680/170498071 [============>.................] - ETA: 11s
 75325440/170498071 [============>.................] - ETA: 11s
 76029952/170498071 [============>.................] - ETA: 11s
 76816384/170498071 [============>.................] - ETA: 10s
 77045760/170498071 [============>.................] - ETA: 10s
 77799424/170498071 [============>.................] - ETA: 10s
 78012416/170498071 [============>.................] - ETA: 10s
 78749696/170498071 [============>.................] - ETA: 10s
 79011840/170498071 [============>.................] - ETA: 10s
 79716352/170498071 [=============>................] - ETA: 10s
 80519168/170498071 [=============>................] - ETA: 10s
 80748544/170498071 [=============>................] - ETA: 10s
 81502208/170498071 [=============>................] - ETA: 10s
 81715200/170498071 [=============>................] - ETA: 10s
 82452480/170498071 [=============>................] - ETA: 10s
 82714624/170498071 [=============>................] - ETA: 10s
 83419136/170498071 [=============>................] - ETA: 10s
 84221952/170498071 [=============>................] - ETA: 10s
 84451328/170498071 [=============>................] - ETA: 10s
 85204992/170498071 [=============>................] - ETA: 9s 
 85401600/170498071 [==============>...............] - ETA: 9s
 86155264/170498071 [==============>...............] - ETA: 9s
 86368256/170498071 [==============>...............] - ETA: 9s
 87121920/170498071 [==============>...............] - ETA: 9s
 87941120/170498071 [==============>...............] - ETA: 9s
 88154112/170498071 [==============>...............] - ETA: 9s
 88891392/170498071 [==============>...............] - ETA: 9s
 89104384/170498071 [==============>...............] - ETA: 9s
 89858048/170498071 [==============>...............] - ETA: 9s
 90071040/170498071 [==============>...............] - ETA: 9s
 90824704/170498071 [==============>...............] - ETA: 9s
 91037696/170498071 [===============>..............] - ETA: 9s
 91774976/170498071 [===============>..............] - ETA: 9s
 92610560/170498071 [===============>..............] - ETA: 8s
 92790784/170498071 [===============>..............] - ETA: 8s
 93560832/170498071 [===============>..............] - ETA: 8s
 93773824/170498071 [===============>..............] - ETA: 8s
 94543872/170498071 [===============>..............] - ETA: 8s
 94773248/170498071 [===============>..............] - ETA: 8s
 95526912/170498071 [===============>..............] - ETA: 8s
 96362496/170498071 [===============>..............] - ETA: 8s
 96542720/170498071 [===============>..............] - ETA: 8s
 97214464/170498071 [================>.............] - ETA: 8s
 97542144/170498071 [================>.............] - ETA: 8s
 98230272/170498071 [================>.............] - ETA: 8s
 98492416/170498071 [================>.............] - ETA: 8s
 99180544/170498071 [================>.............] - ETA: 8s
 99475456/170498071 [================>.............] - ETA: 8s
100229120/170498071 [================>.............] - ETA: 8s
100458496/170498071 [================>.............] - ETA: 8s
101195776/170498071 [================>.............] - ETA: 7s
101736448/170498071 [================>.............] - ETA: 7s
102195200/170498071 [================>.............] - ETA: 7s
102719488/170498071 [=================>............] - ETA: 7s
103178240/170498071 [=================>............] - ETA: 7s
103735296/170498071 [=================>............] - ETA: 7s
104161280/170498071 [=================>............] - ETA: 7s
104718336/170498071 [=================>............] - ETA: 7s
105127936/170498071 [=================>............] - ETA: 7s
105717760/170498071 [=================>............] - ETA: 7s
106127360/170498071 [=================>............] - ETA: 7s
106717184/170498071 [=================>............] - ETA: 7s
107126784/170498071 [=================>............] - ETA: 7s
107732992/170498071 [=================>............] - ETA: 7s
108109824/170498071 [==================>...........] - ETA: 7s
108716032/170498071 [==================>...........] - ETA: 7s
109109248/170498071 [==================>...........] - ETA: 6s
109731840/170498071 [==================>...........] - ETA: 6s
110108672/170498071 [==================>...........] - ETA: 6s
110780416/170498071 [==================>...........] - ETA: 6s
111124480/170498071 [==================>...........] - ETA: 6s
111763456/170498071 [==================>...........] - ETA: 6s
112123904/170498071 [==================>...........] - ETA: 6s
112779264/170498071 [==================>...........] - ETA: 6s
113139712/170498071 [==================>...........] - ETA: 6s
113778688/170498071 [===================>..........] - ETA: 6s
114270208/170498071 [===================>..........] - ETA: 6s
114778112/170498071 [===================>..........] - ETA: 6s
115318784/170498071 [===================>..........] - ETA: 6s
115826688/170498071 [===================>..........] - ETA: 6s
116350976/170498071 [===================>..........] - ETA: 6s
116858880/170498071 [===================>..........] - ETA: 6s
117383168/170498071 [===================>..........] - ETA: 6s
117874688/170498071 [===================>..........] - ETA: 5s
118407168/170498071 [===================>..........] - ETA: 5s
118890496/170498071 [===================>..........] - ETA: 5s
119414784/170498071 [====================>.........] - ETA: 5s
119939072/170498071 [====================>.........] - ETA: 5s
120438784/170498071 [====================>.........] - ETA: 5s
120938496/170498071 [====================>.........] - ETA: 5s
121479168/170498071 [====================>.........] - ETA: 5s
121970688/170498071 [====================>.........] - ETA: 5s
122503168/170498071 [====================>.........] - ETA: 5s
123002880/170498071 [====================>.........] - ETA: 5s
123592704/170498071 [====================>.........] - ETA: 5s
124100608/170498071 [====================>.........] - ETA: 5s
124674048/170498071 [====================>.........] - ETA: 5s
125149184/170498071 [=====================>........] - ETA: 5s
125755392/170498071 [=====================>........] - ETA: 5s
126246912/170498071 [=====================>........] - ETA: 4s
126853120/170498071 [=====================>........] - ETA: 4s
127344640/170498071 [=====================>........] - ETA: 4s
128016384/170498071 [=====================>........] - ETA: 4s
128475136/170498071 [=====================>........] - ETA: 4s
129130496/170498071 [=====================>........] - ETA: 4s
129556480/170498071 [=====================>........] - ETA: 4s
130244608/170498071 [=====================>........] - ETA: 4s
130736128/170498071 [======================>.......] - ETA: 4s
131358720/170498071 [======================>.......] - ETA: 4s
131883008/170498071 [======================>.......] - ETA: 4s
132456448/170498071 [======================>.......] - ETA: 4s
132997120/170498071 [======================>.......] - ETA: 4s
133570560/170498071 [======================>.......] - ETA: 4s
134144000/170498071 [======================>.......] - ETA: 4s
134701056/170498071 [======================>.......] - ETA: 3s
135258112/170498071 [======================>.......] - ETA: 3s
135847936/170498071 [======================>.......] - ETA: 3s
136437760/170498071 [=======================>......] - ETA: 3s
136978432/170498071 [=======================>......] - ETA: 3s
137568256/170498071 [=======================>......] - ETA: 3s
138141696/170498071 [=======================>......] - ETA: 3s
138698752/170498071 [=======================>......] - ETA: 3s
139272192/170498071 [=======================>......] - ETA: 3s
139862016/170498071 [=======================>......] - ETA: 3s
140435456/170498071 [=======================>......] - ETA: 3s
140992512/170498071 [=======================>......] - ETA: 3s
141615104/170498071 [=======================>......] - ETA: 3s
142172160/170498071 [========================>.....] - ETA: 3s
142811136/170498071 [========================>.....] - ETA: 3s
143351808/170498071 [========================>.....] - ETA: 2s
144007168/170498071 [========================>.....] - ETA: 2s
144564224/170498071 [========================>.....] - ETA: 2s
145219584/170498071 [========================>.....] - ETA: 2s
145760256/170498071 [========================>.....] - ETA: 2s
146399232/170498071 [========================>.....] - ETA: 2s
146956288/170498071 [========================>.....] - ETA: 2s
147595264/170498071 [========================>.....] - ETA: 2s
148185088/170498071 [=========================>....] - ETA: 2s
148807680/170498071 [=========================>....] - ETA: 2s
149413888/170498071 [=========================>....] - ETA: 2s
150052864/170498071 [=========================>....] - ETA: 2s
150675456/170498071 [=========================>....] - ETA: 2s
151298048/170498071 [=========================>....] - ETA: 2s
151937024/170498071 [=========================>....] - ETA: 2s
152559616/170498071 [=========================>....] - ETA: 1s
153198592/170498071 [=========================>....] - ETA: 1s
153837568/170498071 [==========================>...] - ETA: 1s
154492928/170498071 [==========================>...] - ETA: 1s
155131904/170498071 [==========================>...] - ETA: 1s
155770880/170498071 [==========================>...] - ETA: 1s
156442624/170498071 [==========================>...] - ETA: 1s
157081600/170498071 [==========================>...] - ETA: 1s
157720576/170498071 [==========================>...] - ETA: 1s
158375936/170498071 [==========================>...] - ETA: 1s
159047680/170498071 [==========================>...] - ETA: 1s
159735808/170498071 [===========================>..] - ETA: 1s
160407552/170498071 [===========================>..] - ETA: 1s
161095680/170498071 [===========================>..] - ETA: 0s
161783808/170498071 [===========================>..] - ETA: 0s
162439168/170498071 [===========================>..] - ETA: 0s
163160064/170498071 [===========================>..] - ETA: 0s
163815424/170498071 [===========================>..] - ETA: 0s
164536320/170498071 [===========================>..] - ETA: 0s
165191680/170498071 [============================>.] - ETA: 0s
165928960/170498071 [============================>.] - ETA: 0s
166600704/170498071 [============================>.] - ETA: 0s
167337984/170498071 [============================>.] - ETA: 0s
168034304/170498071 [============================>.] - ETA: 0s
168747008/170498071 [============================>.] - ETA: 0s
169467904/170498071 [============================>.] - ETA: 0s
170221568/170498071 [============================>.] - ETA: 0s
170500096/170498071 [==============================] - 18s 0us/step

3. Create a quantized Keras VGG model

A Keras model based on the VGG architecture is instantiated with quantized weights and activations.

This model relies only on FullyConnected and Convolutional layers:

  • all the layers have 2-bit weights,

  • all the layers have 2-bit activations.

This model therefore satisfies the Akida NSoC requirements.

This section goes as follows:

  • 3.A - Instantiate a quantized Keras VGG model according to above specifications and load pre-trained weights** that performs 91 % accuracy on the test dataset.

  • 3.B - Check performance on the test set.

3.A Instantiate Keras model

The vgg_cifar10 function returns a VGG Keras model with custom quantized layers (see quantization_layers.py in the CNN2SNN module).

Note

The pre-trained weights which are loaded in the section 3.B corresponds to the quantization parameters in the next cell. If you want to modify some of these parameters, you must re-train the model and save the weights.

Pre-trained weights were obtained after a series of training episodes, starting from unconstrained float weights and activations and ending with quantized 2-bits weights and activations.

For the first training episode, we train the model with unconstrained float weights and activations for 1000 epochs.

For the subsequent training episodes, we start from the weights trained in the previous episode, progressively reducing the bitwidth of activations, then weights. We also stop the episode when the training loss has stopped decreasing for 20 epochs.

The table below summarizes the results obtained when preparing the weights stored under http://data.brainchip.com/models/vgg/:

Episode

Weights Quant.

Activ. Quant.

Accuracy

Epochs

1

N/A

N/A

93.23 %

1000

2

4 bits

4 bits

93.15 %

35

3

3 bits

4 bits

92.65 %

40

4

3 bits

3 bits

92.01 %

83

5

2 bits

3 bits

90.45 %

100

6

2 bits

2 bits

90.59 %

40

Please refer to CNN conversion tutorial and/or the CNN2SNN toolkit documentation for flow and training steps details.

# Instantiate the quantized model
model_keras = vgg_cifar10_pretrained()
model_keras.summary()

Out:

Downloading data from http://data.brainchip.com/models/vgg/vgg_cifar10_wq2_aq2.hdf5

    8192/87690296 [..............................] - ETA: 3:15
   73728/87690296 [..............................] - ETA: 1:29
  270336/87690296 [..............................] - ETA: 43s 
  466944/87690296 [..............................] - ETA: 35s
  663552/87690296 [..............................] - ETA: 32s
  860160/87690296 [..............................] - ETA: 31s
 1056768/87690296 [..............................] - ETA: 30s
 1253376/87690296 [..............................] - ETA: 29s
 1449984/87690296 [..............................] - ETA: 28s
 1646592/87690296 [..............................] - ETA: 28s
 1843200/87690296 [..............................] - ETA: 28s
 2039808/87690296 [..............................] - ETA: 27s
 2236416/87690296 [..............................] - ETA: 27s
 2433024/87690296 [..............................] - ETA: 27s
 2629632/87690296 [..............................] - ETA: 27s
 2826240/87690296 [..............................] - ETA: 26s
 3022848/87690296 [>.............................] - ETA: 26s
 3219456/87690296 [>.............................] - ETA: 26s
 3416064/87690296 [>.............................] - ETA: 26s
 3612672/87690296 [>.............................] - ETA: 26s
 3809280/87690296 [>.............................] - ETA: 26s
 4005888/87690296 [>.............................] - ETA: 26s
 4202496/87690296 [>.............................] - ETA: 25s
 4399104/87690296 [>.............................] - ETA: 25s
 4595712/87690296 [>.............................] - ETA: 25s
 4792320/87690296 [>.............................] - ETA: 25s
 4988928/87690296 [>.............................] - ETA: 25s
 5185536/87690296 [>.............................] - ETA: 25s
 5382144/87690296 [>.............................] - ETA: 25s
 5578752/87690296 [>.............................] - ETA: 25s
 5775360/87690296 [>.............................] - ETA: 25s
 5971968/87690296 [=>............................] - ETA: 25s
 6168576/87690296 [=>............................] - ETA: 24s
 6365184/87690296 [=>............................] - ETA: 24s
 6561792/87690296 [=>............................] - ETA: 24s
 6758400/87690296 [=>............................] - ETA: 24s
 6955008/87690296 [=>............................] - ETA: 24s
 7151616/87690296 [=>............................] - ETA: 24s
 7348224/87690296 [=>............................] - ETA: 24s
 7544832/87690296 [=>............................] - ETA: 24s
 7741440/87690296 [=>............................] - ETA: 24s
 7938048/87690296 [=>............................] - ETA: 24s
 8134656/87690296 [=>............................] - ETA: 24s
 8331264/87690296 [=>............................] - ETA: 24s
 8527872/87690296 [=>............................] - ETA: 24s
 8724480/87690296 [=>............................] - ETA: 24s
 8921088/87690296 [==>...........................] - ETA: 23s
 9117696/87690296 [==>...........................] - ETA: 23s
 9314304/87690296 [==>...........................] - ETA: 23s
 9510912/87690296 [==>...........................] - ETA: 23s
 9707520/87690296 [==>...........................] - ETA: 23s
 9904128/87690296 [==>...........................] - ETA: 23s
10100736/87690296 [==>...........................] - ETA: 23s
10297344/87690296 [==>...........................] - ETA: 23s
10493952/87690296 [==>...........................] - ETA: 23s
10690560/87690296 [==>...........................] - ETA: 23s
10887168/87690296 [==>...........................] - ETA: 23s
11083776/87690296 [==>...........................] - ETA: 23s
11280384/87690296 [==>...........................] - ETA: 23s
11476992/87690296 [==>...........................] - ETA: 23s
11673600/87690296 [==>...........................] - ETA: 23s
11870208/87690296 [===>..........................] - ETA: 22s
12066816/87690296 [===>..........................] - ETA: 22s
12263424/87690296 [===>..........................] - ETA: 22s
12460032/87690296 [===>..........................] - ETA: 22s
12656640/87690296 [===>..........................] - ETA: 22s
12853248/87690296 [===>..........................] - ETA: 22s
13049856/87690296 [===>..........................] - ETA: 22s
13246464/87690296 [===>..........................] - ETA: 22s
13443072/87690296 [===>..........................] - ETA: 22s
13639680/87690296 [===>..........................] - ETA: 22s
13836288/87690296 [===>..........................] - ETA: 22s
14032896/87690296 [===>..........................] - ETA: 22s
14229504/87690296 [===>..........................] - ETA: 22s
14426112/87690296 [===>..........................] - ETA: 22s
14622720/87690296 [====>.........................] - ETA: 22s
14819328/87690296 [====>.........................] - ETA: 21s
15015936/87690296 [====>.........................] - ETA: 21s
15212544/87690296 [====>.........................] - ETA: 21s
15409152/87690296 [====>.........................] - ETA: 21s
15605760/87690296 [====>.........................] - ETA: 21s
15802368/87690296 [====>.........................] - ETA: 21s
15998976/87690296 [====>.........................] - ETA: 21s
16195584/87690296 [====>.........................] - ETA: 21s
16392192/87690296 [====>.........................] - ETA: 21s
16588800/87690296 [====>.........................] - ETA: 21s
16785408/87690296 [====>.........................] - ETA: 21s
16982016/87690296 [====>.........................] - ETA: 21s
17178624/87690296 [====>.........................] - ETA: 21s
17375232/87690296 [====>.........................] - ETA: 21s
17571840/87690296 [=====>........................] - ETA: 21s
17768448/87690296 [=====>........................] - ETA: 21s
17965056/87690296 [=====>........................] - ETA: 20s
18161664/87690296 [=====>........................] - ETA: 20s
18358272/87690296 [=====>........................] - ETA: 20s
18554880/87690296 [=====>........................] - ETA: 20s
18751488/87690296 [=====>........................] - ETA: 20s
18948096/87690296 [=====>........................] - ETA: 20s
19144704/87690296 [=====>........................] - ETA: 20s
19341312/87690296 [=====>........................] - ETA: 20s
19537920/87690296 [=====>........................] - ETA: 20s
19734528/87690296 [=====>........................] - ETA: 20s
19931136/87690296 [=====>........................] - ETA: 20s
20127744/87690296 [=====>........................] - ETA: 20s
20324352/87690296 [=====>........................] - ETA: 20s
20520960/87690296 [======>.......................] - ETA: 20s
20717568/87690296 [======>.......................] - ETA: 20s
20914176/87690296 [======>.......................] - ETA: 20s
21110784/87690296 [======>.......................] - ETA: 20s
21307392/87690296 [======>.......................] - ETA: 19s
21504000/87690296 [======>.......................] - ETA: 19s
21700608/87690296 [======>.......................] - ETA: 19s
21897216/87690296 [======>.......................] - ETA: 19s
22093824/87690296 [======>.......................] - ETA: 19s
22290432/87690296 [======>.......................] - ETA: 19s
22487040/87690296 [======>.......................] - ETA: 19s
22683648/87690296 [======>.......................] - ETA: 19s
22880256/87690296 [======>.......................] - ETA: 19s
23076864/87690296 [======>.......................] - ETA: 19s
23273472/87690296 [======>.......................] - ETA: 19s
23470080/87690296 [=======>......................] - ETA: 19s
23666688/87690296 [=======>......................] - ETA: 19s
23863296/87690296 [=======>......................] - ETA: 19s
24059904/87690296 [=======>......................] - ETA: 19s
24256512/87690296 [=======>......................] - ETA: 19s
24453120/87690296 [=======>......................] - ETA: 18s
24649728/87690296 [=======>......................] - ETA: 18s
24846336/87690296 [=======>......................] - ETA: 18s
25042944/87690296 [=======>......................] - ETA: 18s
25239552/87690296 [=======>......................] - ETA: 18s
25436160/87690296 [=======>......................] - ETA: 18s
25632768/87690296 [=======>......................] - ETA: 18s
25829376/87690296 [=======>......................] - ETA: 18s
26025984/87690296 [=======>......................] - ETA: 18s
26222592/87690296 [=======>......................] - ETA: 18s
26419200/87690296 [========>.....................] - ETA: 18s
26615808/87690296 [========>.....................] - ETA: 18s
26812416/87690296 [========>.....................] - ETA: 18s
27009024/87690296 [========>.....................] - ETA: 18s
27205632/87690296 [========>.....................] - ETA: 18s
27402240/87690296 [========>.....................] - ETA: 18s
27598848/87690296 [========>.....................] - ETA: 18s
27795456/87690296 [========>.....................] - ETA: 17s
27992064/87690296 [========>.....................] - ETA: 17s
28188672/87690296 [========>.....................] - ETA: 17s
28385280/87690296 [========>.....................] - ETA: 17s
28581888/87690296 [========>.....................] - ETA: 17s
28778496/87690296 [========>.....................] - ETA: 17s
28975104/87690296 [========>.....................] - ETA: 17s
29171712/87690296 [========>.....................] - ETA: 17s
29368320/87690296 [=========>....................] - ETA: 17s
29564928/87690296 [=========>....................] - ETA: 17s
29761536/87690296 [=========>....................] - ETA: 17s
29958144/87690296 [=========>....................] - ETA: 17s
30154752/87690296 [=========>....................] - ETA: 17s
30351360/87690296 [=========>....................] - ETA: 17s
30547968/87690296 [=========>....................] - ETA: 17s
30744576/87690296 [=========>....................] - ETA: 17s
30941184/87690296 [=========>....................] - ETA: 17s
31137792/87690296 [=========>....................] - ETA: 16s
31334400/87690296 [=========>....................] - ETA: 16s
31531008/87690296 [=========>....................] - ETA: 16s
31727616/87690296 [=========>....................] - ETA: 16s
31924224/87690296 [=========>....................] - ETA: 16s
32120832/87690296 [=========>....................] - ETA: 16s
32317440/87690296 [==========>...................] - ETA: 16s
32514048/87690296 [==========>...................] - ETA: 16s
32710656/87690296 [==========>...................] - ETA: 16s
32907264/87690296 [==========>...................] - ETA: 16s
33103872/87690296 [==========>...................] - ETA: 16s
33300480/87690296 [==========>...................] - ETA: 16s
33497088/87690296 [==========>...................] - ETA: 16s
33693696/87690296 [==========>...................] - ETA: 16s
33890304/87690296 [==========>...................] - ETA: 16s
34086912/87690296 [==========>...................] - ETA: 16s
34283520/87690296 [==========>...................] - ETA: 16s
34480128/87690296 [==========>...................] - ETA: 15s
34676736/87690296 [==========>...................] - ETA: 15s
34873344/87690296 [==========>...................] - ETA: 15s
35069952/87690296 [==========>...................] - ETA: 15s
35266560/87690296 [===========>..................] - ETA: 15s
35463168/87690296 [===========>..................] - ETA: 15s
35659776/87690296 [===========>..................] - ETA: 15s
35856384/87690296 [===========>..................] - ETA: 15s
36052992/87690296 [===========>..................] - ETA: 15s
36249600/87690296 [===========>..................] - ETA: 15s
36446208/87690296 [===========>..................] - ETA: 15s
36642816/87690296 [===========>..................] - ETA: 15s
36839424/87690296 [===========>..................] - ETA: 15s
37036032/87690296 [===========>..................] - ETA: 15s
37232640/87690296 [===========>..................] - ETA: 15s
37429248/87690296 [===========>..................] - ETA: 15s
37625856/87690296 [===========>..................] - ETA: 15s
37822464/87690296 [===========>..................] - ETA: 14s
38019072/87690296 [============>.................] - ETA: 14s
38215680/87690296 [============>.................] - ETA: 14s
38412288/87690296 [============>.................] - ETA: 14s
38608896/87690296 [============>.................] - ETA: 14s
38805504/87690296 [============>.................] - ETA: 14s
39002112/87690296 [============>.................] - ETA: 14s
39198720/87690296 [============>.................] - ETA: 14s
39395328/87690296 [============>.................] - ETA: 14s
39591936/87690296 [============>.................] - ETA: 14s
39788544/87690296 [============>.................] - ETA: 14s
39985152/87690296 [============>.................] - ETA: 14s
40181760/87690296 [============>.................] - ETA: 14s
40378368/87690296 [============>.................] - ETA: 14s
40574976/87690296 [============>.................] - ETA: 14s
40771584/87690296 [============>.................] - ETA: 14s
40968192/87690296 [=============>................] - ETA: 13s
41164800/87690296 [=============>................] - ETA: 13s
41361408/87690296 [=============>................] - ETA: 13s
41558016/87690296 [=============>................] - ETA: 13s
41754624/87690296 [=============>................] - ETA: 13s
41951232/87690296 [=============>................] - ETA: 13s
42147840/87690296 [=============>................] - ETA: 13s
42344448/87690296 [=============>................] - ETA: 13s
42541056/87690296 [=============>................] - ETA: 13s
42737664/87690296 [=============>................] - ETA: 13s
42934272/87690296 [=============>................] - ETA: 13s
43130880/87690296 [=============>................] - ETA: 13s
43327488/87690296 [=============>................] - ETA: 13s
43524096/87690296 [=============>................] - ETA: 13s
43720704/87690296 [=============>................] - ETA: 13s
43917312/87690296 [==============>...............] - ETA: 13s
44113920/87690296 [==============>...............] - ETA: 13s
44310528/87690296 [==============>...............] - ETA: 12s
44507136/87690296 [==============>...............] - ETA: 12s
44703744/87690296 [==============>...............] - ETA: 12s
44900352/87690296 [==============>...............] - ETA: 12s
45096960/87690296 [==============>...............] - ETA: 12s
45293568/87690296 [==============>...............] - ETA: 12s
45490176/87690296 [==============>...............] - ETA: 12s
45686784/87690296 [==============>...............] - ETA: 12s
45883392/87690296 [==============>...............] - ETA: 12s
46080000/87690296 [==============>...............] - ETA: 12s
46276608/87690296 [==============>...............] - ETA: 12s
46473216/87690296 [==============>...............] - ETA: 12s
46669824/87690296 [==============>...............] - ETA: 12s
46866432/87690296 [===============>..............] - ETA: 12s
47063040/87690296 [===============>..............] - ETA: 12s
47259648/87690296 [===============>..............] - ETA: 12s
47456256/87690296 [===============>..............] - ETA: 12s
47652864/87690296 [===============>..............] - ETA: 11s
47849472/87690296 [===============>..............] - ETA: 11s
48046080/87690296 [===============>..............] - ETA: 11s
48242688/87690296 [===============>..............] - ETA: 11s
48439296/87690296 [===============>..............] - ETA: 11s
48635904/87690296 [===============>..............] - ETA: 11s
48832512/87690296 [===============>..............] - ETA: 11s
49029120/87690296 [===============>..............] - ETA: 11s
49225728/87690296 [===============>..............] - ETA: 11s
49422336/87690296 [===============>..............] - ETA: 11s
49618944/87690296 [===============>..............] - ETA: 11s
49815552/87690296 [================>.............] - ETA: 11s
50012160/87690296 [================>.............] - ETA: 11s
50208768/87690296 [================>.............] - ETA: 11s
50405376/87690296 [================>.............] - ETA: 11s
50601984/87690296 [================>.............] - ETA: 11s
50798592/87690296 [================>.............] - ETA: 11s
50995200/87690296 [================>.............] - ETA: 10s
51191808/87690296 [================>.............] - ETA: 10s
51388416/87690296 [================>.............] - ETA: 10s
51585024/87690296 [================>.............] - ETA: 10s
51781632/87690296 [================>.............] - ETA: 10s
51978240/87690296 [================>.............] - ETA: 10s
52174848/87690296 [================>.............] - ETA: 10s
52371456/87690296 [================>.............] - ETA: 10s
52568064/87690296 [================>.............] - ETA: 10s
52764672/87690296 [=================>............] - ETA: 10s
52961280/87690296 [=================>............] - ETA: 10s
53157888/87690296 [=================>............] - ETA: 10s
53354496/87690296 [=================>............] - ETA: 10s
53551104/87690296 [=================>............] - ETA: 10s
53747712/87690296 [=================>............] - ETA: 10s
53944320/87690296 [=================>............] - ETA: 10s
54140928/87690296 [=================>............] - ETA: 10s
54337536/87690296 [=================>............] - ETA: 9s 
54534144/87690296 [=================>............] - ETA: 9s
54730752/87690296 [=================>............] - ETA: 9s
54927360/87690296 [=================>............] - ETA: 9s
55123968/87690296 [=================>............] - ETA: 9s
55320576/87690296 [=================>............] - ETA: 9s
55517184/87690296 [=================>............] - ETA: 9s
55713792/87690296 [==================>...........] - ETA: 9s
55910400/87690296 [==================>...........] - ETA: 9s
56107008/87690296 [==================>...........] - ETA: 9s
56303616/87690296 [==================>...........] - ETA: 9s
56500224/87690296 [==================>...........] - ETA: 9s
56696832/87690296 [==================>...........] - ETA: 9s
56893440/87690296 [==================>...........] - ETA: 9s
57090048/87690296 [==================>...........] - ETA: 9s
57286656/87690296 [==================>...........] - ETA: 9s
57483264/87690296 [==================>...........] - ETA: 9s
57679872/87690296 [==================>...........] - ETA: 8s
57876480/87690296 [==================>...........] - ETA: 8s
58073088/87690296 [==================>...........] - ETA: 8s
58269696/87690296 [==================>...........] - ETA: 8s
58466304/87690296 [===================>..........] - ETA: 8s
58662912/87690296 [===================>..........] - ETA: 8s
58859520/87690296 [===================>..........] - ETA: 8s
59056128/87690296 [===================>..........] - ETA: 8s
59252736/87690296 [===================>..........] - ETA: 8s
59449344/87690296 [===================>..........] - ETA: 8s
59645952/87690296 [===================>..........] - ETA: 8s
59842560/87690296 [===================>..........] - ETA: 8s
60039168/87690296 [===================>..........] - ETA: 8s
60235776/87690296 [===================>..........] - ETA: 8s
60432384/87690296 [===================>..........] - ETA: 8s
60628992/87690296 [===================>..........] - ETA: 8s
60825600/87690296 [===================>..........] - ETA: 8s
61022208/87690296 [===================>..........] - ETA: 7s
61218816/87690296 [===================>..........] - ETA: 7s
61415424/87690296 [====================>.........] - ETA: 7s
61612032/87690296 [====================>.........] - ETA: 7s
61808640/87690296 [====================>.........] - ETA: 7s
62005248/87690296 [====================>.........] - ETA: 7s
62201856/87690296 [====================>.........] - ETA: 7s
62398464/87690296 [====================>.........] - ETA: 7s
62595072/87690296 [====================>.........] - ETA: 7s
62791680/87690296 [====================>.........] - ETA: 7s
62988288/87690296 [====================>.........] - ETA: 7s
63184896/87690296 [====================>.........] - ETA: 7s
63381504/87690296 [====================>.........] - ETA: 7s
63578112/87690296 [====================>.........] - ETA: 7s
63774720/87690296 [====================>.........] - ETA: 7s
63971328/87690296 [====================>.........] - ETA: 7s
64167936/87690296 [====================>.........] - ETA: 7s
64364544/87690296 [=====================>........] - ETA: 6s
64561152/87690296 [=====================>........] - ETA: 6s
64757760/87690296 [=====================>........] - ETA: 6s
64954368/87690296 [=====================>........] - ETA: 6s
65150976/87690296 [=====================>........] - ETA: 6s
65347584/87690296 [=====================>........] - ETA: 6s
65544192/87690296 [=====================>........] - ETA: 6s
65740800/87690296 [=====================>........] - ETA: 6s
65937408/87690296 [=====================>........] - ETA: 6s
66134016/87690296 [=====================>........] - ETA: 6s
66330624/87690296 [=====================>........] - ETA: 6s
66527232/87690296 [=====================>........] - ETA: 6s
66723840/87690296 [=====================>........] - ETA: 6s
66920448/87690296 [=====================>........] - ETA: 6s
67117056/87690296 [=====================>........] - ETA: 6s
67313664/87690296 [======================>.......] - ETA: 6s
67510272/87690296 [======================>.......] - ETA: 6s
67706880/87690296 [======================>.......] - ETA: 5s
67903488/87690296 [======================>.......] - ETA: 5s
68100096/87690296 [======================>.......] - ETA: 5s
68296704/87690296 [======================>.......] - ETA: 5s
68493312/87690296 [======================>.......] - ETA: 5s
68689920/87690296 [======================>.......] - ETA: 5s
68886528/87690296 [======================>.......] - ETA: 5s
69083136/87690296 [======================>.......] - ETA: 5s
69279744/87690296 [======================>.......] - ETA: 5s
69476352/87690296 [======================>.......] - ETA: 5s
69672960/87690296 [======================>.......] - ETA: 5s
69869568/87690296 [======================>.......] - ETA: 5s
70066176/87690296 [======================>.......] - ETA: 5s
70262784/87690296 [=======================>......] - ETA: 5s
70459392/87690296 [=======================>......] - ETA: 5s
70656000/87690296 [=======================>......] - ETA: 5s
70852608/87690296 [=======================>......] - ETA: 5s
71049216/87690296 [=======================>......] - ETA: 4s
71245824/87690296 [=======================>......] - ETA: 4s
71442432/87690296 [=======================>......] - ETA: 4s
71639040/87690296 [=======================>......] - ETA: 4s
71835648/87690296 [=======================>......] - ETA: 4s
72032256/87690296 [=======================>......] - ETA: 4s
72228864/87690296 [=======================>......] - ETA: 4s
72425472/87690296 [=======================>......] - ETA: 4s
72622080/87690296 [=======================>......] - ETA: 4s
72818688/87690296 [=======================>......] - ETA: 4s
73015296/87690296 [=======================>......] - ETA: 4s
73211904/87690296 [========================>.....] - ETA: 4s
73408512/87690296 [========================>.....] - ETA: 4s
73605120/87690296 [========================>.....] - ETA: 4s
73801728/87690296 [========================>.....] - ETA: 4s
73998336/87690296 [========================>.....] - ETA: 4s
74194944/87690296 [========================>.....] - ETA: 4s
74391552/87690296 [========================>.....] - ETA: 3s
74588160/87690296 [========================>.....] - ETA: 3s
74784768/87690296 [========================>.....] - ETA: 3s
74981376/87690296 [========================>.....] - ETA: 3s
75177984/87690296 [========================>.....] - ETA: 3s
75374592/87690296 [========================>.....] - ETA: 3s
75571200/87690296 [========================>.....] - ETA: 3s
75767808/87690296 [========================>.....] - ETA: 3s
75964416/87690296 [========================>.....] - ETA: 3s
76161024/87690296 [=========================>....] - ETA: 3s
76357632/87690296 [=========================>....] - ETA: 3s
76554240/87690296 [=========================>....] - ETA: 3s
76750848/87690296 [=========================>....] - ETA: 3s
76947456/87690296 [=========================>....] - ETA: 3s
77144064/87690296 [=========================>....] - ETA: 3s
77340672/87690296 [=========================>....] - ETA: 3s
77537280/87690296 [=========================>....] - ETA: 3s
77733888/87690296 [=========================>....] - ETA: 2s
77930496/87690296 [=========================>....] - ETA: 2s
78127104/87690296 [=========================>....] - ETA: 2s
78323712/87690296 [=========================>....] - ETA: 2s
78520320/87690296 [=========================>....] - ETA: 2s
78716928/87690296 [=========================>....] - ETA: 2s
78913536/87690296 [=========================>....] - ETA: 2s
79110144/87690296 [==========================>...] - ETA: 2s
79306752/87690296 [==========================>...] - ETA: 2s
79503360/87690296 [==========================>...] - ETA: 2s
79699968/87690296 [==========================>...] - ETA: 2s
79896576/87690296 [==========================>...] - ETA: 2s
80093184/87690296 [==========================>...] - ETA: 2s
80289792/87690296 [==========================>...] - ETA: 2s
80486400/87690296 [==========================>...] - ETA: 2s
80683008/87690296 [==========================>...] - ETA: 2s
80879616/87690296 [==========================>...] - ETA: 2s
81076224/87690296 [==========================>...] - ETA: 1s
81272832/87690296 [==========================>...] - ETA: 1s
81469440/87690296 [==========================>...] - ETA: 1s
81666048/87690296 [==========================>...] - ETA: 1s
81862656/87690296 [===========================>..] - ETA: 1s
82059264/87690296 [===========================>..] - ETA: 1s
82255872/87690296 [===========================>..] - ETA: 1s
82452480/87690296 [===========================>..] - ETA: 1s
82649088/87690296 [===========================>..] - ETA: 1s
82845696/87690296 [===========================>..] - ETA: 1s
83042304/87690296 [===========================>..] - ETA: 1s
83238912/87690296 [===========================>..] - ETA: 1s
83435520/87690296 [===========================>..] - ETA: 1s
83632128/87690296 [===========================>..] - ETA: 1s
83828736/87690296 [===========================>..] - ETA: 1s
84025344/87690296 [===========================>..] - ETA: 1s
84221952/87690296 [===========================>..] - ETA: 1s
84418560/87690296 [===========================>..] - ETA: 0s
84615168/87690296 [===========================>..] - ETA: 0s
84811776/87690296 [============================>.] - ETA: 0s
85008384/87690296 [============================>.] - ETA: 0s
85204992/87690296 [============================>.] - ETA: 0s
85401600/87690296 [============================>.] - ETA: 0s
85598208/87690296 [============================>.] - ETA: 0s
85794816/87690296 [============================>.] - ETA: 0s
85991424/87690296 [============================>.] - ETA: 0s
86188032/87690296 [============================>.] - ETA: 0s
86384640/87690296 [============================>.] - ETA: 0s
86581248/87690296 [============================>.] - ETA: 0s
86777856/87690296 [============================>.] - ETA: 0s
86974464/87690296 [============================>.] - ETA: 0s
87171072/87690296 [============================>.] - ETA: 0s
87367680/87690296 [============================>.] - ETA: 0s
87564288/87690296 [============================>.] - ETA: 0s
87695360/87690296 [==============================] - 26s 0us/step
Model: "vgg_cifar10"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_12 (InputLayer)        [(None, 32, 32, 3)]       0
_________________________________________________________________
conv_0 (QuantizedConv2D)     (None, 32, 32, 128)       3456
_________________________________________________________________
conv_0_BN (BatchNormalizatio (None, 32, 32, 128)       512
_________________________________________________________________
conv_0_relu (ActivationDiscr (None, 32, 32, 128)       0
_________________________________________________________________
conv_1 (QuantizedConv2D)     (None, 32, 32, 128)       147456
_________________________________________________________________
conv_1_maxpool (MaxPooling2D (None, 16, 16, 128)       0
_________________________________________________________________
conv_1_BN (BatchNormalizatio (None, 16, 16, 128)       512
_________________________________________________________________
conv_1_relu (ActivationDiscr (None, 16, 16, 128)       0
_________________________________________________________________
conv_2 (QuantizedConv2D)     (None, 16, 16, 256)       294912
_________________________________________________________________
conv_2_BN (BatchNormalizatio (None, 16, 16, 256)       1024
_________________________________________________________________
conv_2_relu (ActivationDiscr (None, 16, 16, 256)       0
_________________________________________________________________
conv_3 (QuantizedConv2D)     (None, 16, 16, 256)       589824
_________________________________________________________________
conv_3_maxpool (MaxPooling2D (None, 8, 8, 256)         0
_________________________________________________________________
conv_3_BN (BatchNormalizatio (None, 8, 8, 256)         1024
_________________________________________________________________
conv_3_relu (ActivationDiscr (None, 8, 8, 256)         0
_________________________________________________________________
conv_4 (QuantizedConv2D)     (None, 8, 8, 512)         1179648
_________________________________________________________________
conv_4_BN (BatchNormalizatio (None, 8, 8, 512)         2048
_________________________________________________________________
conv_4_relu (ActivationDiscr (None, 8, 8, 512)         0
_________________________________________________________________
conv_5 (QuantizedConv2D)     (None, 8, 8, 512)         2359296
_________________________________________________________________
conv_5_maxpool (MaxPooling2D (None, 4, 4, 512)         0
_________________________________________________________________
conv_5_BN (BatchNormalizatio (None, 4, 4, 512)         2048
_________________________________________________________________
conv_5_relu (ActivationDiscr (None, 4, 4, 512)         0
_________________________________________________________________
conv_6 (QuantizedConv2D)     (None, 4, 4, 1024)        524288
_________________________________________________________________
conv_6_BN (BatchNormalizatio (None, 4, 4, 1024)        4096
_________________________________________________________________
conv_6_relu (ActivationDiscr (None, 4, 4, 1024)        0
_________________________________________________________________
flatten_4 (Flatten)          (None, 16384)             0
_________________________________________________________________
dense_7 (QuantizedDense)     (None, 1024)              16777216
_________________________________________________________________
dense_7_BN (BatchNormalizati (None, 1024)              4096
_________________________________________________________________
dense_7_relu (ActivationDisc (None, 1024)              0
_________________________________________________________________
dense_8 (QuantizedDense)     (None, 10)                10240
=================================================================
Total params: 21,901,696
Trainable params: 21,894,016
Non-trainable params: 7,680
_________________________________________________________________

3.B Check performance

We check the Keras model accuracy on the first n images of the test set.

The table below summarizes the expected results:

#Images

Accuracy

100

94.00 %

1000

91.80 %

10000

90.59 %

Note

Depending on your hardware setup, the processing time may vary greatly.

num_images = 1000

# Check Model performance
start = timer()
potentials_keras = model_keras.predict(x_test[:num_images])
preds_keras = np.squeeze(np.argmax(potentials_keras, 1))

accuracy = accuracy_score(y_test[:num_images], preds_keras)
print("Accuracy: " + "{0:.2f}".format(100 * accuracy) + "%")
end = timer()
print(f'Keras inference on {num_images} images took {end-start:.2f} s.\n')

Out:

Accuracy: 91.80%
Keras inference on 1000 images took 2.45 s.

4. Conversion to Akida

4.A Convert to Akida model

When converting to an Akida model, we just need to pass the Keras model and the input scaling that was used during training.

# Convert the model
model_akida = convert(model_keras, input_scaling=input_scaling)

4.B Check hardware compliancy

The Model.summary() method provides a detailed description of the Model layers.

It also indicates it they are hardware-compatible (see the HW third column).

model_akida.summary()

Out:

                          Model Summary
_________________________________________________________________
Layer (type)                 Output shape   Kernel shape
=================================================================
conv_0 (InputConvolutional)  [32, 32, 128]  (3, 3, 3, 128)
_________________________________________________________________
conv_1 (Convolutional)       [16, 16, 128]  (3, 3, 128, 128)
_________________________________________________________________
conv_2 (Convolutional)       [16, 16, 256]  (3, 3, 128, 256)
_________________________________________________________________
conv_3 (Convolutional)       [8, 8, 256]    (3, 3, 256, 256)
_________________________________________________________________
conv_4 (Convolutional)       [8, 8, 512]    (3, 3, 256, 512)
_________________________________________________________________
conv_5 (Convolutional)       [4, 4, 512]    (3, 3, 512, 512)
_________________________________________________________________
conv_6 (Convolutional)       [4, 4, 1024]   (1, 1, 512, 1024)
_________________________________________________________________
dense_7 (FullyConnected)     [1, 1, 1024]   (1, 1, 16384, 1024)
_________________________________________________________________
dense_8 (FullyConnected)     [1, 1, 10]     (1, 1, 1024, 10)
_________________________________________________________________
Input shape: 32, 32, 3
Backend type: Software - 1.8.5

4.C Check performance

We check the Akida model accuracy on the first n images of the test set.

The table below summarizes the expected results:

#Images

Accuracy

100

93.00 %

1000

92.10 %

10000

90.67 %

Due to the conversion process, the predictions may be slightly different between the original Keras model and Akida on some specific images.

This explains why when testing on a limited number of images the accuracy numbers between Keras and Akida may be quite different. On the full test set however, the two models accuracies are almost identical.

Note

Depending on your hardware setup, the processing time may vary greatly.

num_images = 1000

# Check Model performance
start = timer()
results = model_akida.predict(raw_x_test[:num_images])
accuracy = accuracy_score(y_test[:num_images], results)

print("Accuracy: " + "{0:.2f}".format(100 * accuracy) + "%")
end = timer()
print(f'Akida inference on {num_images} images took {end-start:.2f} s.\n')

# For non-regression purpose
if num_images == 1000:
    assert accuracy == 0.921

Out:

Accuracy: 92.10%
Akida inference on 1000 images took 2.73 s.
# Print model statistics
print("Model statistics")
stats = model_akida.get_statistics()
model_akida.predict(raw_x_test[:20])
for _, stat in stats.items():
    print(stat)

Out:

Model statistics
Layer (type)                  output sparsity
conv_0 (InputConvolutional)   0.71
Layer (type)                  input sparsity      output sparsity     ops
conv_1 (Convolutional)        0.71                0.80                43491283
Layer (type)                  input sparsity      output sparsity     ops
conv_2 (Convolutional)        0.80                0.81                15272986
Layer (type)                  input sparsity      output sparsity     ops
conv_3 (Convolutional)        0.81                0.84                29060928
Layer (type)                  input sparsity      output sparsity     ops
conv_4 (Convolutional)        0.84                0.89                12118579
Layer (type)                  input sparsity      output sparsity     ops
conv_5 (Convolutional)        0.89                0.87                16452403
Layer (type)                  input sparsity      output sparsity     ops
conv_6 (Convolutional)        0.87                0.87                1082214
Layer (type)                  input sparsity      output sparsity     ops
dense_7 (FullyConnected)      0.87                0.84                2225613
Layer (type)                  input sparsity      output sparsity     ops
dense_8 (FullyConnected)      0.84                0.00                1658

5. Create a quantized Keras DS-CNN model

A DS-CNN Keras model based on the MobileNets architecture is instantiated with quantized weights and activations.

This model relies on a first Convolutional layer followed by several Separable Convolutional layers:

  • all the layers have 4-bit weights,

  • all the layers have 4-bit activations.

This model therefore satisfies the Akida NSoC requirements.

This section goes as follows:

  • 5.A - Instantiate a quantized Keras model according to above specifications

  • 5.B - Check performance on the test set.

5.A Instantiate Keras DS-CNN model

The ds_cnn_cifar10 function returns a MobileNet-like Keras model with custom quantized layers (see quantization_layers.py in the CNN2SNN module).

Note

The pre-trained weights which are loaded in the section 3.B corresponds to the quantization parameters in the next cell. If you want to modify some of these parameters, you must re-train the model and save the weights.

Pre-trained weights were obtained after two training episodes:

  • first, we train the model with unconstrained float weights and activations for 1000 epochs,

  • then, we tune the model with quantized weights initialized from those trained in the previous episode.

We stop the second training episode when the training loss has stopped decreasing for 20 epochs.

The table below summarizes the results obtained when preparing the weights stored under http://data.brainchip.com/models/ds_cnn/:

Episode

Weights Quant.

Activ. Quant.

Accuracy

Epochs

1

N/A

N/A

93.49 %

1000

2

4 bits

4 bits

93.07 %

44

Please refer to CNN conversion tutorial and/or the CNN2SNN toolkit documentation for flow and training steps details.

# Use a quantized model with pretrained quantized weights (93.07% accuracy)
model_keras = ds_cnn_cifar10_pretrained()
model_keras.summary()

Out:

Downloading data from http://data.brainchip.com/models/ds_cnn/ds_cnn_cifar10_wq4_aq4.hdf5

    8192/10835640 [..............................] - ETA: 24s
   73728/10835640 [..............................] - ETA: 10s
  270336/10835640 [..............................] - ETA: 5s 
  466944/10835640 [>.............................] - ETA: 4s
  663552/10835640 [>.............................] - ETA: 3s
  860160/10835640 [=>............................] - ETA: 3s
 1056768/10835640 [=>............................] - ETA: 3s
 1253376/10835640 [==>...........................] - ETA: 3s
 1449984/10835640 [===>..........................] - ETA: 3s
 1646592/10835640 [===>..........................] - ETA: 3s
 1843200/10835640 [====>.........................] - ETA: 2s
 2039808/10835640 [====>.........................] - ETA: 2s
 2236416/10835640 [=====>........................] - ETA: 2s
 2433024/10835640 [=====>........................] - ETA: 2s
 2629632/10835640 [======>.......................] - ETA: 2s
 2826240/10835640 [======>.......................] - ETA: 2s
 3022848/10835640 [=======>......................] - ETA: 2s
 3219456/10835640 [=======>......................] - ETA: 2s
 3416064/10835640 [========>.....................] - ETA: 2s
 3612672/10835640 [=========>....................] - ETA: 2s
 3809280/10835640 [=========>....................] - ETA: 2s
 4005888/10835640 [==========>...................] - ETA: 2s
 4202496/10835640 [==========>...................] - ETA: 2s
 4399104/10835640 [===========>..................] - ETA: 1s
 4595712/10835640 [===========>..................] - ETA: 1s
 4792320/10835640 [============>.................] - ETA: 1s
 4988928/10835640 [============>.................] - ETA: 1s
 5185536/10835640 [=============>................] - ETA: 1s
 5382144/10835640 [=============>................] - ETA: 1s
 5578752/10835640 [==============>...............] - ETA: 1s
 5775360/10835640 [==============>...............] - ETA: 1s
 5971968/10835640 [===============>..............] - ETA: 1s
 6168576/10835640 [================>.............] - ETA: 1s
 6365184/10835640 [================>.............] - ETA: 1s
 6561792/10835640 [=================>............] - ETA: 1s
 6758400/10835640 [=================>............] - ETA: 1s
 6955008/10835640 [==================>...........] - ETA: 1s
 7151616/10835640 [==================>...........] - ETA: 1s
 7348224/10835640 [===================>..........] - ETA: 1s
 7544832/10835640 [===================>..........] - ETA: 1s
 7741440/10835640 [====================>.........] - ETA: 0s
 7938048/10835640 [====================>.........] - ETA: 0s
 8134656/10835640 [=====================>........] - ETA: 0s
 8331264/10835640 [======================>.......] - ETA: 0s
 8527872/10835640 [======================>.......] - ETA: 0s
 8724480/10835640 [=======================>......] - ETA: 0s
 8921088/10835640 [=======================>......] - ETA: 0s
 9117696/10835640 [========================>.....] - ETA: 0s
 9314304/10835640 [========================>.....] - ETA: 0s
 9510912/10835640 [=========================>....] - ETA: 0s
 9707520/10835640 [=========================>....] - ETA: 0s
 9904128/10835640 [==========================>...] - ETA: 0s
10100736/10835640 [==========================>...] - ETA: 0s
10297344/10835640 [===========================>..] - ETA: 0s
10493952/10835640 [============================>.] - ETA: 0s
10690560/10835640 [============================>.] - ETA: 0s
10838016/10835640 [==============================] - 3s 0us/step
Model: "ds_cnn_cifar10"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_8 (InputLayer)         [(None, 32, 32, 3)]       0
_________________________________________________________________
conv_0 (QuantizedConv2D)     (None, 32, 32, 128)       3456
_________________________________________________________________
conv_0_BN (BatchNormalizatio (None, 32, 32, 128)       512
_________________________________________________________________
conv_0_relu (ActivationDiscr (None, 32, 32, 128)       0
_________________________________________________________________
separable_1 (QuantizedSepara (None, 32, 32, 128)       17536
_________________________________________________________________
separable_1_BN (BatchNormali (None, 32, 32, 128)       512
_________________________________________________________________
separable_1_relu (Activation (None, 32, 32, 128)       0
_________________________________________________________________
separable_2 (QuantizedSepara (None, 32, 32, 256)       33920
_________________________________________________________________
separable_2_BN (BatchNormali (None, 32, 32, 256)       1024
_________________________________________________________________
separable_2_relu (Activation (None, 32, 32, 256)       0
_________________________________________________________________
separable_3 (QuantizedSepara (None, 32, 32, 256)       67840
_________________________________________________________________
separable_3_maxpool (MaxPool (None, 16, 16, 256)       0
_________________________________________________________________
separable_3_BN (BatchNormali (None, 16, 16, 256)       1024
_________________________________________________________________
separable_3_relu (Activation (None, 16, 16, 256)       0
_________________________________________________________________
separable_4 (QuantizedSepara (None, 16, 16, 512)       133376
_________________________________________________________________
separable_4_BN (BatchNormali (None, 16, 16, 512)       2048
_________________________________________________________________
separable_4_relu (Activation (None, 16, 16, 512)       0
_________________________________________________________________
separable_5 (QuantizedSepara (None, 16, 16, 512)       266752
_________________________________________________________________
separable_5_maxpool (MaxPool (None, 8, 8, 512)         0
_________________________________________________________________
separable_5_BN (BatchNormali (None, 8, 8, 512)         2048
_________________________________________________________________
separable_5_relu (Activation (None, 8, 8, 512)         0
_________________________________________________________________
separable_6 (QuantizedSepara (None, 8, 8, 512)         266752
_________________________________________________________________
separable_6_BN (BatchNormali (None, 8, 8, 512)         2048
_________________________________________________________________
separable_6_relu (Activation (None, 8, 8, 512)         0
_________________________________________________________________
separable_7 (QuantizedSepara (None, 8, 8, 512)         266752
_________________________________________________________________
separable_7_maxpool (MaxPool (None, 4, 4, 512)         0
_________________________________________________________________
separable_7_BN (BatchNormali (None, 4, 4, 512)         2048
_________________________________________________________________
separable_7_relu (Activation (None, 4, 4, 512)         0
_________________________________________________________________
separable_8 (QuantizedSepara (None, 4, 4, 1024)        528896
_________________________________________________________________
separable_8_BN (BatchNormali (None, 4, 4, 1024)        4096
_________________________________________________________________
separable_8_relu (Activation (None, 4, 4, 1024)        0
_________________________________________________________________
separable_9 (QuantizedSepara (None, 4, 4, 1024)        1057792
_________________________________________________________________
separable_9_BN (BatchNormali (None, 4, 4, 1024)        4096
_________________________________________________________________
separable_9_relu (Activation (None, 4, 4, 1024)        0
_________________________________________________________________
separable_10 (QuantizedSepar (None, 4, 4, 10)          19456
_________________________________________________________________
separable_10_global_avg (Glo (None, 10)                0
=================================================================
Total params: 2,681,984
Trainable params: 2,672,256
Non-trainable params: 9,728
_________________________________________________________________

5.B Check performance

We check the Keras model accuracy on the first n images of the test set.

The table below summarizes the expected results:

#Images

Accuracy

100

95.00 %

1000

93.10 %

10000

93.07 %

Note

Depending on your hardware setup, the processing time may vary greatly.

num_images = 1000

# Check Model performance
start = timer()
potentials_keras = model_keras.predict(x_test[:num_images])
preds_keras = np.squeeze(np.argmax(potentials_keras, 1))

accuracy = accuracy_score(y_test[:num_images], preds_keras)
print("Accuracy: " + "{0:.2f}".format(100 * accuracy) + "%")
end = timer()
print(f'Keras inference on {num_images} images took {end-start:.2f} s.\n')

Out:

Accuracy: 93.00%
Keras inference on 1000 images took 2.39 s.

6. Conversion to Akida

6.A Convert to Akida model

When converting to an Akida model, we just need to pass the Keras model and the input scaling that was used during training.

model_akida = convert(model_keras, input_scaling=input_scaling)

6.B Check hardware compliancy

The Model.summary() method provides a detailed description of the Model layers.

It also indicates it they are hardware-compatible (see the HW third column).

model_akida.summary()

Out:

                                       Model Summary
___________________________________________________________________________________________
Layer (type)                           Output shape   Kernel shape
===========================================================================================
conv_0 (InputConvolutional)            [32, 32, 128]  (3, 3, 3, 128)
___________________________________________________________________________________________
separable_1 (SeparableConvolutional)   [32, 32, 128]  (3, 3, 128, 1), (1, 1, 128, 128)
___________________________________________________________________________________________
separable_2 (SeparableConvolutional)   [32, 32, 256]  (3, 3, 128, 1), (1, 1, 128, 256)
___________________________________________________________________________________________
separable_3 (SeparableConvolutional)   [16, 16, 256]  (3, 3, 256, 1), (1, 1, 256, 256)
___________________________________________________________________________________________
separable_4 (SeparableConvolutional)   [16, 16, 512]  (3, 3, 256, 1), (1, 1, 256, 512)
___________________________________________________________________________________________
separable_5 (SeparableConvolutional)   [8, 8, 512]    (3, 3, 512, 1), (1, 1, 512, 512)
___________________________________________________________________________________________
separable_6 (SeparableConvolutional)   [8, 8, 512]    (3, 3, 512, 1), (1, 1, 512, 512)
___________________________________________________________________________________________
separable_7 (SeparableConvolutional)   [4, 4, 512]    (3, 3, 512, 1), (1, 1, 512, 512)
___________________________________________________________________________________________
separable_8 (SeparableConvolutional)   [4, 4, 1024]   (3, 3, 512, 1), (1, 1, 512, 1024)
___________________________________________________________________________________________
separable_9 (SeparableConvolutional)   [4, 4, 1024]   (3, 3, 1024, 1), (1, 1, 1024, 1024)
___________________________________________________________________________________________
separable_10 (SeparableConvolutional)  [1, 1, 10]     (3, 3, 1024, 1), (1, 1, 1024, 10)
___________________________________________________________________________________________
Input shape: 32, 32, 3
Backend type: Software - 1.8.5

6.C Check performance

We check the Akida model accuracy on the first n images of the test set.

The table below summarizes the expected results:

#Images

Accuracy

100

95.00 %

1000

93.10 %

10000

93.22 %

Due to the conversion process, the predictions may be slightly different between the original Keras model and Akida on some specific images.

This explains why when testing on a limited number of images the accuracy numbers between Keras and Akida may be quite different. On the full test set however, the two models accuracies are almost identical.

Note

Depending on your hardware setup, the processing time may vary greatly.

num_images = 1000

# Check Model performance
start = timer()
results = model_akida.predict(raw_x_test[:num_images])
accuracy = accuracy_score(y_test[:num_images], results)

print("Accuracy: " + "{0:.2f}".format(100 * accuracy) + "%")
end = timer()
print(f'Akida inference on {num_images} images took {end-start:.2f} s.\n')

# For non-regression purpose
if num_images == 1000:
    assert accuracy == 0.931

Out:

Accuracy: 93.10%
Akida inference on 1000 images took 7.16 s.
# Print model statistics
print("Model statistics")
stats = model_akida.get_statistics()
model_akida.predict(raw_x_test[:20])
for _, stat in stats.items():
    print(stat)

Out:

Model statistics
Layer (type)                  output sparsity
conv_0 (InputConvolutional)   0.59
Layer (type)                  input sparsity      output sparsity     ops
separable_1 (SeparableConvolu 0.59                0.53                62663175
Layer (type)                  input sparsity      output sparsity     ops
separable_2 (SeparableConvolu 0.53                0.54                143484989
Layer (type)                  input sparsity      output sparsity     ops
separable_3 (SeparableConvolu 0.54                0.61                279008748
Layer (type)                  input sparsity      output sparsity     ops
separable_4 (SeparableConvolu 0.61                0.65                118130331
Layer (type)                  input sparsity      output sparsity     ops
separable_5 (SeparableConvolu 0.65                0.70                214518748
Layer (type)                  input sparsity      output sparsity     ops
separable_6 (SeparableConvolu 0.70                0.68                44972119
Layer (type)                  input sparsity      output sparsity     ops
separable_7 (SeparableConvolu 0.68                0.75                48254114
Layer (type)                  input sparsity      output sparsity     ops
separable_8 (SeparableConvolu 0.75                0.84                18696769
Layer (type)                  input sparsity      output sparsity     ops
separable_9 (SeparableConvolu 0.84                0.84                24647816
Layer (type)                  input sparsity      output sparsity     ops
separable_10 (SeparableConvol 0.84                0.00                260459

6D. Show predictions for a random image

import matplotlib.pyplot as plt
import matplotlib.lines as lines
import matplotlib.patches as patches

label_names = [
    'airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse',
    'ship', 'truck'
]

# prepare plot
barWidth = 0.75
pause_time = 1

fig = plt.figure(num='CIFAR10 Classification by Akida Execution Engine',
                 figsize=(8, 4))
ax0 = plt.subplot(1, 3, 1)
imgobj = ax0.imshow(np.zeros((32, 32, 3), dtype=np.uint8))
ax0.set_axis_off()
# Results subplots
ax1 = plt.subplot(1, 2, 2)
ax1.xaxis.set_visible(False)
ax0.text(0, 34, 'Actual class:')
actual_class = ax0.text(16, 34, 'None')
ax0.text(0, 37, 'Predicted class:')
predicted_class = ax0.text(20, 37, 'None')

# Take a random test image
i = np.random.randint(y_test.shape[0])

true_idx = int(y_test[i])
pot = model_akida.evaluate(np.expand_dims(raw_x_test[i], axis=0)).squeeze()

rpot = np.arange(len(pot))
ax1.barh(rpot, pot, height=barWidth)
ax1.set_yticks(rpot - 0.07 * barWidth)
ax1.set_yticklabels(label_names)
predicted_idx = pot.argmax()
imgobj.set_data(raw_x_test[i])
if predicted_idx == true_idx:
    ax1.get_children()[predicted_idx].set_color('g')
else:
    ax1.get_children()[predicted_idx].set_color('r')
actual_class.set_text(label_names[true_idx])
predicted_class.set_text(label_names[predicted_idx])
ax1.set_title('Akida\'s predictions')
plt.show()
../_images/sphx_glr_plot_vgg_ds_cnn_cifar10_001.png

Total running time of the script: ( 1 minutes 6.134 seconds)

Gallery generated by Sphinx-Gallery