Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a TF-Keras floating-point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original TF-Keras floating-point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf_keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import tensorflow as tf
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from tf_keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, height, width, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
16384/11490434 [..............................] - ETA: 42s
49152/11490434 [..............................] - ETA: 33s
81920/11490434 [..............................] - ETA: 31s
147456/11490434 [..............................] - ETA: 22s
212992/11490434 [..............................] - ETA: 18s
278528/11490434 [..............................] - ETA: 15s
344064/11490434 [..............................] - ETA: 14s
409600/11490434 [>.............................] - ETA: 13s
507904/11490434 [>.............................] - ETA: 12s
557056/11490434 [>.............................] - ETA: 12s
638976/11490434 [>.............................] - ETA: 11s
720896/11490434 [>.............................] - ETA: 10s
786432/11490434 [=>............................] - ETA: 10s
851968/11490434 [=>............................] - ETA: 10s
950272/11490434 [=>............................] - ETA: 9s
999424/11490434 [=>............................] - ETA: 10s
1081344/11490434 [=>............................] - ETA: 9s
1163264/11490434 [==>...........................] - ETA: 9s
1228800/11490434 [==>...........................] - ETA: 9s
1294336/11490434 [==>...........................] - ETA: 9s
1392640/11490434 [==>...........................] - ETA: 8s
1441792/11490434 [==>...........................] - ETA: 8s
1523712/11490434 [==>...........................] - ETA: 8s
1605632/11490434 [===>..........................] - ETA: 8s
1671168/11490434 [===>..........................] - ETA: 8s
1736704/11490434 [===>..........................] - ETA: 8s
1835008/11490434 [===>..........................] - ETA: 8s
1884160/11490434 [===>..........................] - ETA: 8s
1966080/11490434 [====>.........................] - ETA: 8s
2048000/11490434 [====>.........................] - ETA: 7s
2121728/11490434 [====>.........................] - ETA: 7s
2179072/11490434 [====>.........................] - ETA: 7s
2277376/11490434 [====>.........................] - ETA: 7s
2326528/11490434 [=====>........................] - ETA: 7s
2408448/11490434 [=====>........................] - ETA: 7s
2490368/11490434 [=====>........................] - ETA: 7s
2555904/11490434 [=====>........................] - ETA: 7s
2621440/11490434 [=====>........................] - ETA: 7s
2719744/11490434 [======>.......................] - ETA: 7s
2768896/11490434 [======>.......................] - ETA: 7s
2850816/11490434 [======>.......................] - ETA: 7s
2932736/11490434 [======>.......................] - ETA: 7s
2998272/11490434 [======>.......................] - ETA: 6s
3063808/11490434 [======>.......................] - ETA: 6s
3162112/11490434 [=======>......................] - ETA: 6s
3211264/11490434 [=======>......................] - ETA: 6s
3293184/11490434 [=======>......................] - ETA: 6s
3391488/11490434 [=======>......................] - ETA: 6s
3506176/11490434 [========>.....................] - ETA: 6s
3571712/11490434 [========>.....................] - ETA: 6s
3620864/11490434 [========>.....................] - ETA: 6s
3686400/11490434 [========>.....................] - ETA: 6s
3735552/11490434 [========>.....................] - ETA: 6s
3833856/11490434 [=========>....................] - ETA: 6s
3948544/11490434 [=========>....................] - ETA: 6s
4063232/11490434 [=========>....................] - ETA: 6s
4177920/11490434 [=========>....................] - ETA: 6s
4276224/11490434 [==========>...................] - ETA: 6s
4390912/11490434 [==========>...................] - ETA: 5s
4505600/11490434 [==========>...................] - ETA: 5s
4620288/11490434 [===========>..................] - ETA: 5s
4718592/11490434 [===========>..................] - ETA: 5s
4833280/11490434 [===========>..................] - ETA: 5s
4947968/11490434 [===========>..................] - ETA: 5s
5062656/11490434 [============>.................] - ETA: 5s
5160960/11490434 [============>.................] - ETA: 5s
5275648/11490434 [============>.................] - ETA: 5s
5390336/11490434 [=============>................] - ETA: 4s
5505024/11490434 [=============>................] - ETA: 4s
5603328/11490434 [=============>................] - ETA: 4s
5718016/11490434 [=============>................] - ETA: 4s
5832704/11490434 [==============>...............] - ETA: 4s
5947392/11490434 [==============>...............] - ETA: 4s
6045696/11490434 [==============>...............] - ETA: 4s
6160384/11490434 [===============>..............] - ETA: 4s
6275072/11490434 [===============>..............] - ETA: 4s
6389760/11490434 [===============>..............] - ETA: 4s
6488064/11490434 [===============>..............] - ETA: 4s
6602752/11490434 [================>.............] - ETA: 3s
6717440/11490434 [================>.............] - ETA: 3s
6832128/11490434 [================>.............] - ETA: 3s
6930432/11490434 [=================>............] - ETA: 3s
7045120/11490434 [=================>............] - ETA: 3s
7159808/11490434 [=================>............] - ETA: 3s
7274496/11490434 [=================>............] - ETA: 3s
7372800/11490434 [==================>...........] - ETA: 3s
7487488/11490434 [==================>...........] - ETA: 3s
7602176/11490434 [==================>...........] - ETA: 3s
7716864/11490434 [===================>..........] - ETA: 2s
7815168/11490434 [===================>..........] - ETA: 2s
7929856/11490434 [===================>..........] - ETA: 2s
8044544/11490434 [====================>.........] - ETA: 2s
8159232/11490434 [====================>.........] - ETA: 2s
8257536/11490434 [====================>.........] - ETA: 2s
8372224/11490434 [====================>.........] - ETA: 2s
8486912/11490434 [=====================>........] - ETA: 2s
8601600/11490434 [=====================>........] - ETA: 2s
8699904/11490434 [=====================>........] - ETA: 2s
8814592/11490434 [======================>.......] - ETA: 2s
8929280/11490434 [======================>.......] - ETA: 2s
9043968/11490434 [======================>.......] - ETA: 1s
9142272/11490434 [======================>.......] - ETA: 1s
9256960/11490434 [=======================>......] - ETA: 1s
9371648/11490434 [=======================>......] - ETA: 1s
9486336/11490434 [=======================>......] - ETA: 1s
9584640/11490434 [========================>.....] - ETA: 1s
9699328/11490434 [========================>.....] - ETA: 1s
9814016/11490434 [========================>.....] - ETA: 1s
9928704/11490434 [========================>.....] - ETA: 1s
10027008/11490434 [=========================>....] - ETA: 1s
10141696/11490434 [=========================>....] - ETA: 1s
10256384/11490434 [=========================>....] - ETA: 0s
10371072/11490434 [==========================>...] - ETA: 0s
10469376/11490434 [==========================>...] - ETA: 0s
10584064/11490434 [==========================>...] - ETA: 0s
10633216/11490434 [==========================>...] - ETA: 0s
10698752/11490434 [==========================>...] - ETA: 0s
10747904/11490434 [===========================>..] - ETA: 0s
10878976/11490434 [===========================>..] - ETA: 0s
10993664/11490434 [===========================>..] - ETA: 0s
11091968/11490434 [===========================>..] - ETA: 0s
11206656/11490434 [============================>.] - ETA: 0s
11321344/11490434 [============================>.] - ETA: 0s
11436032/11490434 [============================>.] - ETA: 0s
11490434/11490434 [==============================] - 9s 1us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard TF-Keras CNN model.
import tf_keras as keras
model_keras = keras.models.Sequential([
keras.layers.Input(shape=(28, 28, 1), name="input", dtype=tf.uint8),
keras.layers.Rescaling(1. / 255),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from tf_keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:13:08 - loss: 2.6926 - accuracy: 0.0312
23/1688 [..............................] - ETA: 3s - loss: 1.5301 - accuracy: 0.5231
42/1688 [..............................] - ETA: 4s - loss: 1.1325 - accuracy: 0.6600
64/1688 [>.............................] - ETA: 4s - loss: 0.9061 - accuracy: 0.7271
85/1688 [>.............................] - ETA: 3s - loss: 0.7911 - accuracy: 0.7629
107/1688 [>.............................] - ETA: 3s - loss: 0.6984 - accuracy: 0.7897
129/1688 [=>............................] - ETA: 3s - loss: 0.6311 - accuracy: 0.8096
151/1688 [=>............................] - ETA: 3s - loss: 0.5817 - accuracy: 0.8228
174/1688 [==>...........................] - ETA: 3s - loss: 0.5380 - accuracy: 0.8364
197/1688 [==>...........................] - ETA: 3s - loss: 0.4995 - accuracy: 0.8468
220/1688 [==>...........................] - ETA: 3s - loss: 0.4755 - accuracy: 0.8551
243/1688 [===>..........................] - ETA: 3s - loss: 0.4475 - accuracy: 0.8639
265/1688 [===>..........................] - ETA: 3s - loss: 0.4267 - accuracy: 0.8715
287/1688 [====>.........................] - ETA: 3s - loss: 0.4127 - accuracy: 0.8754
310/1688 [====>.........................] - ETA: 3s - loss: 0.3980 - accuracy: 0.8799
332/1688 [====>.........................] - ETA: 3s - loss: 0.3844 - accuracy: 0.8844
354/1688 [=====>........................] - ETA: 3s - loss: 0.3694 - accuracy: 0.8892
377/1688 [=====>........................] - ETA: 3s - loss: 0.3563 - accuracy: 0.8938
400/1688 [======>.......................] - ETA: 2s - loss: 0.3480 - accuracy: 0.8963
422/1688 [======>.......................] - ETA: 2s - loss: 0.3383 - accuracy: 0.8991
444/1688 [======>.......................] - ETA: 2s - loss: 0.3281 - accuracy: 0.9020
467/1688 [=======>......................] - ETA: 2s - loss: 0.3203 - accuracy: 0.9046
489/1688 [=======>......................] - ETA: 2s - loss: 0.3134 - accuracy: 0.9068
511/1688 [========>.....................] - ETA: 2s - loss: 0.3067 - accuracy: 0.9089
534/1688 [========>.....................] - ETA: 2s - loss: 0.3012 - accuracy: 0.9106
556/1688 [========>.....................] - ETA: 2s - loss: 0.2935 - accuracy: 0.9129
578/1688 [=========>....................] - ETA: 2s - loss: 0.2882 - accuracy: 0.9146
600/1688 [=========>....................] - ETA: 2s - loss: 0.2828 - accuracy: 0.9162
621/1688 [==========>...................] - ETA: 2s - loss: 0.2788 - accuracy: 0.9171
643/1688 [==========>...................] - ETA: 2s - loss: 0.2737 - accuracy: 0.9188
664/1688 [==========>...................] - ETA: 2s - loss: 0.2686 - accuracy: 0.9204
687/1688 [===========>..................] - ETA: 2s - loss: 0.2647 - accuracy: 0.9212
710/1688 [===========>..................] - ETA: 2s - loss: 0.2605 - accuracy: 0.9224
732/1688 [============>.................] - ETA: 2s - loss: 0.2561 - accuracy: 0.9238
755/1688 [============>.................] - ETA: 2s - loss: 0.2527 - accuracy: 0.9249
778/1688 [============>.................] - ETA: 2s - loss: 0.2495 - accuracy: 0.9259
801/1688 [=============>................] - ETA: 2s - loss: 0.2464 - accuracy: 0.9270
822/1688 [=============>................] - ETA: 1s - loss: 0.2422 - accuracy: 0.9284
845/1688 [==============>...............] - ETA: 1s - loss: 0.2393 - accuracy: 0.9292
868/1688 [==============>...............] - ETA: 1s - loss: 0.2356 - accuracy: 0.9301
891/1688 [==============>...............] - ETA: 1s - loss: 0.2328 - accuracy: 0.9310
914/1688 [===============>..............] - ETA: 1s - loss: 0.2295 - accuracy: 0.9320
937/1688 [===============>..............] - ETA: 1s - loss: 0.2258 - accuracy: 0.9330
960/1688 [================>.............] - ETA: 1s - loss: 0.2229 - accuracy: 0.9335
982/1688 [================>.............] - ETA: 1s - loss: 0.2201 - accuracy: 0.9343
1004/1688 [================>.............] - ETA: 1s - loss: 0.2178 - accuracy: 0.9352
1027/1688 [=================>............] - ETA: 1s - loss: 0.2152 - accuracy: 0.9360
1049/1688 [=================>............] - ETA: 1s - loss: 0.2126 - accuracy: 0.9367
1072/1688 [==================>...........] - ETA: 1s - loss: 0.2108 - accuracy: 0.9372
1095/1688 [==================>...........] - ETA: 1s - loss: 0.2083 - accuracy: 0.9377
1118/1688 [==================>...........] - ETA: 1s - loss: 0.2060 - accuracy: 0.9385
1141/1688 [===================>..........] - ETA: 1s - loss: 0.2038 - accuracy: 0.9392
1163/1688 [===================>..........] - ETA: 1s - loss: 0.2021 - accuracy: 0.9398
1185/1688 [====================>.........] - ETA: 1s - loss: 0.2012 - accuracy: 0.9400
1208/1688 [====================>.........] - ETA: 1s - loss: 0.1991 - accuracy: 0.9405
1230/1688 [====================>.........] - ETA: 1s - loss: 0.1972 - accuracy: 0.9410
1253/1688 [=====================>........] - ETA: 0s - loss: 0.1953 - accuracy: 0.9415
1276/1688 [=====================>........] - ETA: 0s - loss: 0.1939 - accuracy: 0.9418
1299/1688 [======================>.......] - ETA: 0s - loss: 0.1923 - accuracy: 0.9422
1322/1688 [======================>.......] - ETA: 0s - loss: 0.1906 - accuracy: 0.9427
1345/1688 [======================>.......] - ETA: 0s - loss: 0.1893 - accuracy: 0.9431
1368/1688 [=======================>......] - ETA: 0s - loss: 0.1877 - accuracy: 0.9435
1390/1688 [=======================>......] - ETA: 0s - loss: 0.1859 - accuracy: 0.9440
1413/1688 [========================>.....] - ETA: 0s - loss: 0.1838 - accuracy: 0.9446
1435/1688 [========================>.....] - ETA: 0s - loss: 0.1826 - accuracy: 0.9450
1458/1688 [========================>.....] - ETA: 0s - loss: 0.1815 - accuracy: 0.9454
1481/1688 [=========================>....] - ETA: 0s - loss: 0.1804 - accuracy: 0.9457
1504/1688 [=========================>....] - ETA: 0s - loss: 0.1787 - accuracy: 0.9463
1527/1688 [==========================>...] - ETA: 0s - loss: 0.1768 - accuracy: 0.9468
1550/1688 [==========================>...] - ETA: 0s - loss: 0.1758 - accuracy: 0.9472
1573/1688 [==========================>...] - ETA: 0s - loss: 0.1741 - accuracy: 0.9477
1595/1688 [===========================>..] - ETA: 0s - loss: 0.1736 - accuracy: 0.9478
1618/1688 [===========================>..] - ETA: 0s - loss: 0.1723 - accuracy: 0.9481
1640/1688 [============================>.] - ETA: 0s - loss: 0.1715 - accuracy: 0.9484
1663/1688 [============================>.] - ETA: 0s - loss: 0.1700 - accuracy: 0.9488
1685/1688 [============================>.] - ETA: 0s - loss: 0.1698 - accuracy: 0.9489
1688/1688 [==============================] - ETA: 0s - loss: 0.1697 - accuracy: 0.9489
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1697 - accuracy: 0.9489 - val_loss: 0.0727 - val_accuracy: 0.9797
Epoch 2/10
1/1688 [..............................] - ETA: 4s - loss: 0.0121 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0421 - accuracy: 0.9883
46/1688 [..............................] - ETA: 3s - loss: 0.0579 - accuracy: 0.9810
69/1688 [>.............................] - ETA: 3s - loss: 0.0700 - accuracy: 0.9764
92/1688 [>.............................] - ETA: 3s - loss: 0.0748 - accuracy: 0.9752
114/1688 [=>............................] - ETA: 3s - loss: 0.0752 - accuracy: 0.9756
136/1688 [=>............................] - ETA: 3s - loss: 0.0726 - accuracy: 0.9766
159/1688 [=>............................] - ETA: 3s - loss: 0.0720 - accuracy: 0.9764
182/1688 [==>...........................] - ETA: 3s - loss: 0.0724 - accuracy: 0.9754
205/1688 [==>...........................] - ETA: 3s - loss: 0.0739 - accuracy: 0.9753
228/1688 [===>..........................] - ETA: 3s - loss: 0.0730 - accuracy: 0.9757
251/1688 [===>..........................] - ETA: 3s - loss: 0.0736 - accuracy: 0.9755
274/1688 [===>..........................] - ETA: 3s - loss: 0.0751 - accuracy: 0.9754
297/1688 [====>.........................] - ETA: 3s - loss: 0.0753 - accuracy: 0.9756
320/1688 [====>.........................] - ETA: 3s - loss: 0.0779 - accuracy: 0.9744
342/1688 [=====>........................] - ETA: 3s - loss: 0.0772 - accuracy: 0.9751
364/1688 [=====>........................] - ETA: 2s - loss: 0.0783 - accuracy: 0.9750
387/1688 [=====>........................] - ETA: 2s - loss: 0.0776 - accuracy: 0.9754
410/1688 [======>.......................] - ETA: 2s - loss: 0.0778 - accuracy: 0.9755
432/1688 [======>.......................] - ETA: 2s - loss: 0.0777 - accuracy: 0.9753
453/1688 [=======>......................] - ETA: 2s - loss: 0.0779 - accuracy: 0.9753
475/1688 [=======>......................] - ETA: 2s - loss: 0.0778 - accuracy: 0.9757
497/1688 [=======>......................] - ETA: 2s - loss: 0.0782 - accuracy: 0.9757
520/1688 [========>.....................] - ETA: 2s - loss: 0.0773 - accuracy: 0.9761
543/1688 [========>.....................] - ETA: 2s - loss: 0.0771 - accuracy: 0.9762
565/1688 [=========>....................] - ETA: 2s - loss: 0.0767 - accuracy: 0.9762
587/1688 [=========>....................] - ETA: 2s - loss: 0.0767 - accuracy: 0.9763
609/1688 [=========>....................] - ETA: 2s - loss: 0.0772 - accuracy: 0.9761
632/1688 [==========>...................] - ETA: 2s - loss: 0.0767 - accuracy: 0.9762
654/1688 [==========>...................] - ETA: 2s - loss: 0.0765 - accuracy: 0.9761
676/1688 [===========>..................] - ETA: 2s - loss: 0.0756 - accuracy: 0.9763
698/1688 [===========>..................] - ETA: 2s - loss: 0.0747 - accuracy: 0.9765
721/1688 [===========>..................] - ETA: 2s - loss: 0.0748 - accuracy: 0.9764
744/1688 [============>.................] - ETA: 2s - loss: 0.0747 - accuracy: 0.9766
767/1688 [============>.................] - ETA: 2s - loss: 0.0745 - accuracy: 0.9765
789/1688 [=============>................] - ETA: 2s - loss: 0.0748 - accuracy: 0.9767
811/1688 [=============>................] - ETA: 1s - loss: 0.0748 - accuracy: 0.9766
834/1688 [=============>................] - ETA: 1s - loss: 0.0749 - accuracy: 0.9766
856/1688 [==============>...............] - ETA: 1s - loss: 0.0748 - accuracy: 0.9766
879/1688 [==============>...............] - ETA: 1s - loss: 0.0750 - accuracy: 0.9767
902/1688 [===============>..............] - ETA: 1s - loss: 0.0749 - accuracy: 0.9767
925/1688 [===============>..............] - ETA: 1s - loss: 0.0750 - accuracy: 0.9767
948/1688 [===============>..............] - ETA: 1s - loss: 0.0746 - accuracy: 0.9767
970/1688 [================>.............] - ETA: 1s - loss: 0.0750 - accuracy: 0.9765
993/1688 [================>.............] - ETA: 1s - loss: 0.0747 - accuracy: 0.9766
1016/1688 [=================>............] - ETA: 1s - loss: 0.0741 - accuracy: 0.9767
1039/1688 [=================>............] - ETA: 1s - loss: 0.0738 - accuracy: 0.9768
1062/1688 [=================>............] - ETA: 1s - loss: 0.0738 - accuracy: 0.9768
1084/1688 [==================>...........] - ETA: 1s - loss: 0.0737 - accuracy: 0.9767
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0736 - accuracy: 0.9768
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0735 - accuracy: 0.9768
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9770
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9770
1199/1688 [====================>.........] - ETA: 1s - loss: 0.0734 - accuracy: 0.9769
1221/1688 [====================>.........] - ETA: 1s - loss: 0.0728 - accuracy: 0.9770
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9770
1267/1688 [=====================>........] - ETA: 0s - loss: 0.0729 - accuracy: 0.9770
1290/1688 [=====================>........] - ETA: 0s - loss: 0.0733 - accuracy: 0.9769
1312/1688 [======================>.......] - ETA: 0s - loss: 0.0734 - accuracy: 0.9769
1334/1688 [======================>.......] - ETA: 0s - loss: 0.0734 - accuracy: 0.9768
1356/1688 [=======================>......] - ETA: 0s - loss: 0.0731 - accuracy: 0.9769
1378/1688 [=======================>......] - ETA: 0s - loss: 0.0727 - accuracy: 0.9770
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0727 - accuracy: 0.9770
1424/1688 [========================>.....] - ETA: 0s - loss: 0.0724 - accuracy: 0.9770
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0721 - accuracy: 0.9772
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0723 - accuracy: 0.9773
1494/1688 [=========================>....] - ETA: 0s - loss: 0.0723 - accuracy: 0.9774
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0719 - accuracy: 0.9775
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0714 - accuracy: 0.9776
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0714 - accuracy: 0.9776
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0710 - accuracy: 0.9777
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0707 - accuracy: 0.9777
1632/1688 [============================>.] - ETA: 0s - loss: 0.0705 - accuracy: 0.9778
1655/1688 [============================>.] - ETA: 0s - loss: 0.0703 - accuracy: 0.9778
1678/1688 [============================>.] - ETA: 0s - loss: 0.0702 - accuracy: 0.9778
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0700 - accuracy: 0.9779 - val_loss: 0.0611 - val_accuracy: 0.9835
Epoch 3/10
1/1688 [..............................] - ETA: 4s - loss: 0.0221 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0430 - accuracy: 0.9870
46/1688 [..............................] - ETA: 3s - loss: 0.0414 - accuracy: 0.9864
68/1688 [>.............................] - ETA: 3s - loss: 0.0373 - accuracy: 0.9881
91/1688 [>.............................] - ETA: 3s - loss: 0.0410 - accuracy: 0.9880
114/1688 [=>............................] - ETA: 3s - loss: 0.0415 - accuracy: 0.9871
137/1688 [=>............................] - ETA: 3s - loss: 0.0414 - accuracy: 0.9868
159/1688 [=>............................] - ETA: 3s - loss: 0.0416 - accuracy: 0.9868
181/1688 [==>...........................] - ETA: 3s - loss: 0.0435 - accuracy: 0.9857
203/1688 [==>...........................] - ETA: 3s - loss: 0.0424 - accuracy: 0.9855
225/1688 [==>...........................] - ETA: 3s - loss: 0.0459 - accuracy: 0.9851
247/1688 [===>..........................] - ETA: 3s - loss: 0.0476 - accuracy: 0.9847
270/1688 [===>..........................] - ETA: 3s - loss: 0.0499 - accuracy: 0.9844
292/1688 [====>.........................] - ETA: 3s - loss: 0.0494 - accuracy: 0.9845
314/1688 [====>.........................] - ETA: 3s - loss: 0.0490 - accuracy: 0.9848
337/1688 [====>.........................] - ETA: 3s - loss: 0.0498 - accuracy: 0.9848
360/1688 [=====>........................] - ETA: 3s - loss: 0.0487 - accuracy: 0.9849
383/1688 [=====>........................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9849
405/1688 [======>.......................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9846
427/1688 [======>.......................] - ETA: 2s - loss: 0.0502 - accuracy: 0.9845
449/1688 [======>.......................] - ETA: 2s - loss: 0.0495 - accuracy: 0.9849
471/1688 [=======>......................] - ETA: 2s - loss: 0.0494 - accuracy: 0.9846
493/1688 [=======>......................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9847
516/1688 [========>.....................] - ETA: 2s - loss: 0.0492 - accuracy: 0.9849
539/1688 [========>.....................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9851
561/1688 [========>.....................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9848
583/1688 [=========>....................] - ETA: 2s - loss: 0.0495 - accuracy: 0.9847
606/1688 [=========>....................] - ETA: 2s - loss: 0.0495 - accuracy: 0.9845
630/1688 [==========>...................] - ETA: 2s - loss: 0.0508 - accuracy: 0.9842
653/1688 [==========>...................] - ETA: 2s - loss: 0.0507 - accuracy: 0.9842
676/1688 [===========>..................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9845
698/1688 [===========>..................] - ETA: 2s - loss: 0.0506 - accuracy: 0.9846
721/1688 [===========>..................] - ETA: 2s - loss: 0.0513 - accuracy: 0.9842
744/1688 [============>.................] - ETA: 2s - loss: 0.0518 - accuracy: 0.9842
768/1688 [============>.................] - ETA: 2s - loss: 0.0524 - accuracy: 0.9840
792/1688 [=============>................] - ETA: 2s - loss: 0.0524 - accuracy: 0.9840
814/1688 [=============>................] - ETA: 1s - loss: 0.0521 - accuracy: 0.9841
836/1688 [=============>................] - ETA: 1s - loss: 0.0516 - accuracy: 0.9841
858/1688 [==============>...............] - ETA: 1s - loss: 0.0521 - accuracy: 0.9842
880/1688 [==============>...............] - ETA: 1s - loss: 0.0521 - accuracy: 0.9842
902/1688 [===============>..............] - ETA: 1s - loss: 0.0523 - accuracy: 0.9840
925/1688 [===============>..............] - ETA: 1s - loss: 0.0527 - accuracy: 0.9840
947/1688 [===============>..............] - ETA: 1s - loss: 0.0530 - accuracy: 0.9839
970/1688 [================>.............] - ETA: 1s - loss: 0.0530 - accuracy: 0.9840
993/1688 [================>.............] - ETA: 1s - loss: 0.0527 - accuracy: 0.9840
1016/1688 [=================>............] - ETA: 1s - loss: 0.0533 - accuracy: 0.9840
1038/1688 [=================>............] - ETA: 1s - loss: 0.0535 - accuracy: 0.9839
1060/1688 [=================>............] - ETA: 1s - loss: 0.0539 - accuracy: 0.9836
1083/1688 [==================>...........] - ETA: 1s - loss: 0.0542 - accuracy: 0.9835
1106/1688 [==================>...........] - ETA: 1s - loss: 0.0542 - accuracy: 0.9834
1128/1688 [===================>..........] - ETA: 1s - loss: 0.0540 - accuracy: 0.9834
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0535 - accuracy: 0.9836
1173/1688 [===================>..........] - ETA: 1s - loss: 0.0533 - accuracy: 0.9836
1196/1688 [====================>.........] - ETA: 1s - loss: 0.0534 - accuracy: 0.9836
1219/1688 [====================>.........] - ETA: 1s - loss: 0.0532 - accuracy: 0.9836
1241/1688 [=====================>........] - ETA: 1s - loss: 0.0532 - accuracy: 0.9837
1264/1688 [=====================>........] - ETA: 0s - loss: 0.0532 - accuracy: 0.9836
1287/1688 [=====================>........] - ETA: 0s - loss: 0.0538 - accuracy: 0.9833
1309/1688 [======================>.......] - ETA: 0s - loss: 0.0547 - accuracy: 0.9832
1331/1688 [======================>.......] - ETA: 0s - loss: 0.0547 - accuracy: 0.9833
1354/1688 [=======================>......] - ETA: 0s - loss: 0.0544 - accuracy: 0.9834
1376/1688 [=======================>......] - ETA: 0s - loss: 0.0544 - accuracy: 0.9834
1399/1688 [=======================>......] - ETA: 0s - loss: 0.0542 - accuracy: 0.9834
1421/1688 [========================>.....] - ETA: 0s - loss: 0.0545 - accuracy: 0.9834
1443/1688 [========================>.....] - ETA: 0s - loss: 0.0546 - accuracy: 0.9832
1465/1688 [=========================>....] - ETA: 0s - loss: 0.0543 - accuracy: 0.9833
1487/1688 [=========================>....] - ETA: 0s - loss: 0.0542 - accuracy: 0.9833
1510/1688 [=========================>....] - ETA: 0s - loss: 0.0545 - accuracy: 0.9832
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0545 - accuracy: 0.9831
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0546 - accuracy: 0.9832
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0543 - accuracy: 0.9832
1601/1688 [===========================>..] - ETA: 0s - loss: 0.0542 - accuracy: 0.9832
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0542 - accuracy: 0.9833
1647/1688 [============================>.] - ETA: 0s - loss: 0.0538 - accuracy: 0.9834
1670/1688 [============================>.] - ETA: 0s - loss: 0.0536 - accuracy: 0.9834
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0533 - accuracy: 0.9835 - val_loss: 0.0670 - val_accuracy: 0.9815
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0226 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0359 - accuracy: 0.9883
46/1688 [..............................] - ETA: 3s - loss: 0.0312 - accuracy: 0.9891
69/1688 [>.............................] - ETA: 3s - loss: 0.0370 - accuracy: 0.9891
92/1688 [>.............................] - ETA: 3s - loss: 0.0341 - accuracy: 0.9901
114/1688 [=>............................] - ETA: 3s - loss: 0.0348 - accuracy: 0.9896
136/1688 [=>............................] - ETA: 3s - loss: 0.0354 - accuracy: 0.9885
158/1688 [=>............................] - ETA: 3s - loss: 0.0328 - accuracy: 0.9897
180/1688 [==>...........................] - ETA: 3s - loss: 0.0314 - accuracy: 0.9901
203/1688 [==>...........................] - ETA: 3s - loss: 0.0311 - accuracy: 0.9903
225/1688 [==>...........................] - ETA: 3s - loss: 0.0326 - accuracy: 0.9892
247/1688 [===>..........................] - ETA: 3s - loss: 0.0326 - accuracy: 0.9891
270/1688 [===>..........................] - ETA: 3s - loss: 0.0318 - accuracy: 0.9895
292/1688 [====>.........................] - ETA: 3s - loss: 0.0304 - accuracy: 0.9903
314/1688 [====>.........................] - ETA: 3s - loss: 0.0317 - accuracy: 0.9898
337/1688 [====>.........................] - ETA: 3s - loss: 0.0331 - accuracy: 0.9894
360/1688 [=====>........................] - ETA: 3s - loss: 0.0335 - accuracy: 0.9889
382/1688 [=====>........................] - ETA: 2s - loss: 0.0336 - accuracy: 0.9888
405/1688 [======>.......................] - ETA: 2s - loss: 0.0339 - accuracy: 0.9887
427/1688 [======>.......................] - ETA: 2s - loss: 0.0341 - accuracy: 0.9884
450/1688 [======>.......................] - ETA: 2s - loss: 0.0347 - accuracy: 0.9885
473/1688 [=======>......................] - ETA: 2s - loss: 0.0345 - accuracy: 0.9886
496/1688 [=======>......................] - ETA: 2s - loss: 0.0352 - accuracy: 0.9883
519/1688 [========>.....................] - ETA: 2s - loss: 0.0355 - accuracy: 0.9883
542/1688 [========>.....................] - ETA: 2s - loss: 0.0360 - accuracy: 0.9882
564/1688 [=========>....................] - ETA: 2s - loss: 0.0366 - accuracy: 0.9879
587/1688 [=========>....................] - ETA: 2s - loss: 0.0365 - accuracy: 0.9880
610/1688 [=========>....................] - ETA: 2s - loss: 0.0373 - accuracy: 0.9879
633/1688 [==========>...................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9881
655/1688 [==========>...................] - ETA: 2s - loss: 0.0369 - accuracy: 0.9881
678/1688 [===========>..................] - ETA: 2s - loss: 0.0373 - accuracy: 0.9880
700/1688 [===========>..................] - ETA: 2s - loss: 0.0378 - accuracy: 0.9879
722/1688 [===========>..................] - ETA: 2s - loss: 0.0387 - accuracy: 0.9875
745/1688 [============>.................] - ETA: 2s - loss: 0.0393 - accuracy: 0.9873
767/1688 [============>.................] - ETA: 2s - loss: 0.0402 - accuracy: 0.9868
790/1688 [=============>................] - ETA: 2s - loss: 0.0405 - accuracy: 0.9867
813/1688 [=============>................] - ETA: 1s - loss: 0.0404 - accuracy: 0.9869
835/1688 [=============>................] - ETA: 1s - loss: 0.0403 - accuracy: 0.9868
858/1688 [==============>...............] - ETA: 1s - loss: 0.0405 - accuracy: 0.9867
881/1688 [==============>...............] - ETA: 1s - loss: 0.0405 - accuracy: 0.9867
904/1688 [===============>..............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9866
927/1688 [===============>..............] - ETA: 1s - loss: 0.0406 - accuracy: 0.9866
950/1688 [===============>..............] - ETA: 1s - loss: 0.0408 - accuracy: 0.9864
972/1688 [================>.............] - ETA: 1s - loss: 0.0411 - accuracy: 0.9864
995/1688 [================>.............] - ETA: 1s - loss: 0.0411 - accuracy: 0.9864
1017/1688 [=================>............] - ETA: 1s - loss: 0.0412 - accuracy: 0.9863
1039/1688 [=================>............] - ETA: 1s - loss: 0.0413 - accuracy: 0.9862
1062/1688 [=================>............] - ETA: 1s - loss: 0.0415 - accuracy: 0.9862
1085/1688 [==================>...........] - ETA: 1s - loss: 0.0414 - accuracy: 0.9863
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0413 - accuracy: 0.9862
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0416 - accuracy: 0.9862
1155/1688 [===================>..........] - ETA: 1s - loss: 0.0417 - accuracy: 0.9862
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0418 - accuracy: 0.9861
1200/1688 [====================>.........] - ETA: 1s - loss: 0.0418 - accuracy: 0.9861
1224/1688 [====================>.........] - ETA: 1s - loss: 0.0418 - accuracy: 0.9861
1247/1688 [=====================>........] - ETA: 0s - loss: 0.0420 - accuracy: 0.9861
1269/1688 [=====================>........] - ETA: 0s - loss: 0.0417 - accuracy: 0.9861
1291/1688 [=====================>........] - ETA: 0s - loss: 0.0414 - accuracy: 0.9862
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0413 - accuracy: 0.9863
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0412 - accuracy: 0.9864
1359/1688 [=======================>......] - ETA: 0s - loss: 0.0414 - accuracy: 0.9863
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0414 - accuracy: 0.9864
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0415 - accuracy: 0.9863
1429/1688 [========================>.....] - ETA: 0s - loss: 0.0414 - accuracy: 0.9863
1452/1688 [========================>.....] - ETA: 0s - loss: 0.0415 - accuracy: 0.9864
1475/1688 [=========================>....] - ETA: 0s - loss: 0.0416 - accuracy: 0.9863
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0415 - accuracy: 0.9864
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0415 - accuracy: 0.9863
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0420 - accuracy: 0.9862
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0421 - accuracy: 0.9862
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0421 - accuracy: 0.9861
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0423 - accuracy: 0.9861
1633/1688 [============================>.] - ETA: 0s - loss: 0.0421 - accuracy: 0.9862
1656/1688 [============================>.] - ETA: 0s - loss: 0.0420 - accuracy: 0.9862
1678/1688 [============================>.] - ETA: 0s - loss: 0.0418 - accuracy: 0.9862
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0417 - accuracy: 0.9863 - val_loss: 0.0531 - val_accuracy: 0.9860
Epoch 5/10
1/1688 [..............................] - ETA: 3s - loss: 0.0235 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9922
47/1688 [..............................] - ETA: 3s - loss: 0.0226 - accuracy: 0.9907
69/1688 [>.............................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9914
92/1688 [>.............................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9922
115/1688 [=>............................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9918
137/1688 [=>............................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9918
160/1688 [=>............................] - ETA: 3s - loss: 0.0226 - accuracy: 0.9914
184/1688 [==>...........................] - ETA: 3s - loss: 0.0230 - accuracy: 0.9913
207/1688 [==>...........................] - ETA: 3s - loss: 0.0233 - accuracy: 0.9911
230/1688 [===>..........................] - ETA: 3s - loss: 0.0231 - accuracy: 0.9913
252/1688 [===>..........................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9908
274/1688 [===>..........................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9908
297/1688 [====>.........................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9903
320/1688 [====>.........................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9906
342/1688 [=====>........................] - ETA: 3s - loss: 0.0268 - accuracy: 0.9908
365/1688 [=====>........................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9905
387/1688 [=====>........................] - ETA: 2s - loss: 0.0289 - accuracy: 0.9901
409/1688 [======>.......................] - ETA: 2s - loss: 0.0287 - accuracy: 0.9901
431/1688 [======>.......................] - ETA: 2s - loss: 0.0284 - accuracy: 0.9903
453/1688 [=======>......................] - ETA: 2s - loss: 0.0284 - accuracy: 0.9901
476/1688 [=======>......................] - ETA: 2s - loss: 0.0281 - accuracy: 0.9904
499/1688 [=======>......................] - ETA: 2s - loss: 0.0277 - accuracy: 0.9905
521/1688 [========>.....................] - ETA: 2s - loss: 0.0283 - accuracy: 0.9905
543/1688 [========>.....................] - ETA: 2s - loss: 0.0290 - accuracy: 0.9902
565/1688 [=========>....................] - ETA: 2s - loss: 0.0285 - accuracy: 0.9903
587/1688 [=========>....................] - ETA: 2s - loss: 0.0286 - accuracy: 0.9902
610/1688 [=========>....................] - ETA: 2s - loss: 0.0294 - accuracy: 0.9900
633/1688 [==========>...................] - ETA: 2s - loss: 0.0296 - accuracy: 0.9899
655/1688 [==========>...................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9899
677/1688 [===========>..................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9898
699/1688 [===========>..................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9897
722/1688 [===========>..................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9896
744/1688 [============>.................] - ETA: 2s - loss: 0.0310 - accuracy: 0.9895
766/1688 [============>.................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9896
789/1688 [=============>................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9895
813/1688 [=============>................] - ETA: 1s - loss: 0.0307 - accuracy: 0.9895
835/1688 [=============>................] - ETA: 1s - loss: 0.0310 - accuracy: 0.9894
858/1688 [==============>...............] - ETA: 1s - loss: 0.0314 - accuracy: 0.9892
880/1688 [==============>...............] - ETA: 1s - loss: 0.0315 - accuracy: 0.9890
902/1688 [===============>..............] - ETA: 1s - loss: 0.0314 - accuracy: 0.9891
924/1688 [===============>..............] - ETA: 1s - loss: 0.0315 - accuracy: 0.9891
946/1688 [===============>..............] - ETA: 1s - loss: 0.0317 - accuracy: 0.9891
968/1688 [================>.............] - ETA: 1s - loss: 0.0313 - accuracy: 0.9892
991/1688 [================>.............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9893
1013/1688 [=================>............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9893
1036/1688 [=================>............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9892
1058/1688 [=================>............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9892
1081/1688 [==================>...........] - ETA: 1s - loss: 0.0314 - accuracy: 0.9893
1103/1688 [==================>...........] - ETA: 1s - loss: 0.0315 - accuracy: 0.9892
1126/1688 [===================>..........] - ETA: 1s - loss: 0.0313 - accuracy: 0.9893
1149/1688 [===================>..........] - ETA: 1s - loss: 0.0313 - accuracy: 0.9893
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0314 - accuracy: 0.9892
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0316 - accuracy: 0.9892
1215/1688 [====================>.........] - ETA: 1s - loss: 0.0317 - accuracy: 0.9891
1238/1688 [=====================>........] - ETA: 1s - loss: 0.0317 - accuracy: 0.9891
1260/1688 [=====================>........] - ETA: 0s - loss: 0.0317 - accuracy: 0.9892
1283/1688 [=====================>........] - ETA: 0s - loss: 0.0316 - accuracy: 0.9892
1306/1688 [======================>.......] - ETA: 0s - loss: 0.0313 - accuracy: 0.9893
1329/1688 [======================>.......] - ETA: 0s - loss: 0.0315 - accuracy: 0.9893
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0317 - accuracy: 0.9893
1375/1688 [=======================>......] - ETA: 0s - loss: 0.0320 - accuracy: 0.9892
1398/1688 [=======================>......] - ETA: 0s - loss: 0.0320 - accuracy: 0.9892
1421/1688 [========================>.....] - ETA: 0s - loss: 0.0322 - accuracy: 0.9892
1444/1688 [========================>.....] - ETA: 0s - loss: 0.0322 - accuracy: 0.9892
1467/1688 [=========================>....] - ETA: 0s - loss: 0.0322 - accuracy: 0.9892
1490/1688 [=========================>....] - ETA: 0s - loss: 0.0325 - accuracy: 0.9892
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0324 - accuracy: 0.9893
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0323 - accuracy: 0.9893
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0322 - accuracy: 0.9894
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0323 - accuracy: 0.9893
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0323 - accuracy: 0.9894
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0323 - accuracy: 0.9894
1649/1688 [============================>.] - ETA: 0s - loss: 0.0321 - accuracy: 0.9895
1671/1688 [============================>.] - ETA: 0s - loss: 0.0322 - accuracy: 0.9894
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0321 - accuracy: 0.9894 - val_loss: 0.0631 - val_accuracy: 0.9843
Epoch 6/10
1/1688 [..............................] - ETA: 3s - loss: 0.0020 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0240 - accuracy: 0.9891
45/1688 [..............................] - ETA: 3s - loss: 0.0226 - accuracy: 0.9903
67/1688 [>.............................] - ETA: 3s - loss: 0.0270 - accuracy: 0.9902
90/1688 [>.............................] - ETA: 3s - loss: 0.0285 - accuracy: 0.9899
112/1688 [>.............................] - ETA: 3s - loss: 0.0273 - accuracy: 0.9900
135/1688 [=>............................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9900
157/1688 [=>............................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9902
180/1688 [==>...........................] - ETA: 3s - loss: 0.0265 - accuracy: 0.9898
203/1688 [==>...........................] - ETA: 3s - loss: 0.0251 - accuracy: 0.9903
225/1688 [==>...........................] - ETA: 3s - loss: 0.0240 - accuracy: 0.9908
248/1688 [===>..........................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9912
270/1688 [===>..........................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9910
292/1688 [====>.........................] - ETA: 3s - loss: 0.0248 - accuracy: 0.9905
315/1688 [====>.........................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9904
337/1688 [====>.........................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9904
359/1688 [=====>........................] - ETA: 3s - loss: 0.0247 - accuracy: 0.9909
381/1688 [=====>........................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9908
404/1688 [======>.......................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9908
426/1688 [======>.......................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9905
449/1688 [======>.......................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9905
471/1688 [=======>......................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9906
493/1688 [=======>......................] - ETA: 2s - loss: 0.0260 - accuracy: 0.9907
515/1688 [========>.....................] - ETA: 2s - loss: 0.0260 - accuracy: 0.9908
537/1688 [========>.....................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9906
560/1688 [========>.....................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9907
583/1688 [=========>....................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9908
606/1688 [=========>....................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9907
629/1688 [==========>...................] - ETA: 2s - loss: 0.0265 - accuracy: 0.9906
651/1688 [==========>...................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9904
674/1688 [==========>...................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9906
696/1688 [===========>..................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9905
719/1688 [===========>..................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9906
741/1688 [============>.................] - ETA: 2s - loss: 0.0271 - accuracy: 0.9906
763/1688 [============>.................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9907
785/1688 [============>.................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9908
807/1688 [=============>................] - ETA: 2s - loss: 0.0275 - accuracy: 0.9906
830/1688 [=============>................] - ETA: 1s - loss: 0.0277 - accuracy: 0.9905
853/1688 [==============>...............] - ETA: 1s - loss: 0.0277 - accuracy: 0.9905
876/1688 [==============>...............] - ETA: 1s - loss: 0.0281 - accuracy: 0.9904
898/1688 [==============>...............] - ETA: 1s - loss: 0.0289 - accuracy: 0.9902
920/1688 [===============>..............] - ETA: 1s - loss: 0.0288 - accuracy: 0.9903
942/1688 [===============>..............] - ETA: 1s - loss: 0.0288 - accuracy: 0.9901
963/1688 [================>.............] - ETA: 1s - loss: 0.0290 - accuracy: 0.9901
986/1688 [================>.............] - ETA: 1s - loss: 0.0292 - accuracy: 0.9901
1009/1688 [================>.............] - ETA: 1s - loss: 0.0289 - accuracy: 0.9902
1031/1688 [=================>............] - ETA: 1s - loss: 0.0292 - accuracy: 0.9902
1054/1688 [=================>............] - ETA: 1s - loss: 0.0290 - accuracy: 0.9902
1077/1688 [==================>...........] - ETA: 1s - loss: 0.0288 - accuracy: 0.9902
1100/1688 [==================>...........] - ETA: 1s - loss: 0.0287 - accuracy: 0.9903
1122/1688 [==================>...........] - ETA: 1s - loss: 0.0288 - accuracy: 0.9902
1144/1688 [===================>..........] - ETA: 1s - loss: 0.0290 - accuracy: 0.9901
1167/1688 [===================>..........] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0295 - accuracy: 0.9900
1212/1688 [====================>.........] - ETA: 1s - loss: 0.0295 - accuracy: 0.9901
1235/1688 [====================>.........] - ETA: 1s - loss: 0.0293 - accuracy: 0.9901
1257/1688 [=====================>........] - ETA: 0s - loss: 0.0292 - accuracy: 0.9902
1280/1688 [=====================>........] - ETA: 0s - loss: 0.0293 - accuracy: 0.9902
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0295 - accuracy: 0.9900
1325/1688 [======================>.......] - ETA: 0s - loss: 0.0294 - accuracy: 0.9901
1347/1688 [======================>.......] - ETA: 0s - loss: 0.0293 - accuracy: 0.9901
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0299 - accuracy: 0.9899
1392/1688 [=======================>......] - ETA: 0s - loss: 0.0299 - accuracy: 0.9899
1414/1688 [========================>.....] - ETA: 0s - loss: 0.0299 - accuracy: 0.9899
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0299 - accuracy: 0.9898
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0297 - accuracy: 0.9899
1482/1688 [=========================>....] - ETA: 0s - loss: 0.0294 - accuracy: 0.9900
1505/1688 [=========================>....] - ETA: 0s - loss: 0.0295 - accuracy: 0.9900
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0294 - accuracy: 0.9900
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0293 - accuracy: 0.9900
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0296 - accuracy: 0.9900
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0296 - accuracy: 0.9900
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0295 - accuracy: 0.9900
1641/1688 [============================>.] - ETA: 0s - loss: 0.0293 - accuracy: 0.9901
1664/1688 [============================>.] - ETA: 0s - loss: 0.0295 - accuracy: 0.9900
1686/1688 [============================>.] - ETA: 0s - loss: 0.0296 - accuracy: 0.9899
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0296 - accuracy: 0.9899 - val_loss: 0.0688 - val_accuracy: 0.9840
Epoch 7/10
1/1688 [..............................] - ETA: 4s - loss: 0.0480 - accuracy: 0.9688
23/1688 [..............................] - ETA: 3s - loss: 0.0221 - accuracy: 0.9932
46/1688 [..............................] - ETA: 3s - loss: 0.0165 - accuracy: 0.9946
69/1688 [>.............................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9941
92/1688 [>.............................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9946
115/1688 [=>............................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9951
138/1688 [=>............................] - ETA: 3s - loss: 0.0188 - accuracy: 0.9939
162/1688 [=>............................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9934
185/1688 [==>...........................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9929
207/1688 [==>...........................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9929
230/1688 [===>..........................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9932
252/1688 [===>..........................] - ETA: 3s - loss: 0.0204 - accuracy: 0.9936
275/1688 [===>..........................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9930
298/1688 [====>.........................] - ETA: 3s - loss: 0.0204 - accuracy: 0.9930
322/1688 [====>.........................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9923
345/1688 [=====>........................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9926
367/1688 [=====>........................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9923
390/1688 [=====>........................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9925
413/1688 [======>.......................] - ETA: 2s - loss: 0.0207 - accuracy: 0.9926
436/1688 [======>.......................] - ETA: 2s - loss: 0.0213 - accuracy: 0.9925
458/1688 [=======>......................] - ETA: 2s - loss: 0.0210 - accuracy: 0.9926
481/1688 [=======>......................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9927
503/1688 [=======>......................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9927
525/1688 [========>.....................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9927
547/1688 [========>.....................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9927
570/1688 [=========>....................] - ETA: 2s - loss: 0.0207 - accuracy: 0.9929
593/1688 [=========>....................] - ETA: 2s - loss: 0.0207 - accuracy: 0.9929
615/1688 [=========>....................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9930
638/1688 [==========>...................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9931
660/1688 [==========>...................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9933
682/1688 [===========>..................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9933
705/1688 [===========>..................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9933
727/1688 [===========>..................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9933
750/1688 [============>.................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9934
773/1688 [============>.................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9935
796/1688 [=============>................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9935
819/1688 [=============>................] - ETA: 1s - loss: 0.0199 - accuracy: 0.9936
841/1688 [=============>................] - ETA: 1s - loss: 0.0196 - accuracy: 0.9936
864/1688 [==============>...............] - ETA: 1s - loss: 0.0194 - accuracy: 0.9937
886/1688 [==============>...............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9937
909/1688 [===============>..............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9938
931/1688 [===============>..............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9939
953/1688 [===============>..............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9937
975/1688 [================>.............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9937
998/1688 [================>.............] - ETA: 1s - loss: 0.0195 - accuracy: 0.9937
1020/1688 [=================>............] - ETA: 1s - loss: 0.0196 - accuracy: 0.9937
1042/1688 [=================>............] - ETA: 1s - loss: 0.0197 - accuracy: 0.9936
1065/1688 [=================>............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9935
1088/1688 [==================>...........] - ETA: 1s - loss: 0.0200 - accuracy: 0.9935
1110/1688 [==================>...........] - ETA: 1s - loss: 0.0201 - accuracy: 0.9934
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0201 - accuracy: 0.9934
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0200 - accuracy: 0.9933
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0202 - accuracy: 0.9933
1200/1688 [====================>.........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9932
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0208 - accuracy: 0.9930
1246/1688 [=====================>........] - ETA: 0s - loss: 0.0210 - accuracy: 0.9930
1269/1688 [=====================>........] - ETA: 0s - loss: 0.0213 - accuracy: 0.9928
1293/1688 [=====================>........] - ETA: 0s - loss: 0.0213 - accuracy: 0.9928
1315/1688 [======================>.......] - ETA: 0s - loss: 0.0213 - accuracy: 0.9929
1337/1688 [======================>.......] - ETA: 0s - loss: 0.0213 - accuracy: 0.9928
1359/1688 [=======================>......] - ETA: 0s - loss: 0.0216 - accuracy: 0.9928
1382/1688 [=======================>......] - ETA: 0s - loss: 0.0223 - accuracy: 0.9926
1405/1688 [=======================>......] - ETA: 0s - loss: 0.0225 - accuracy: 0.9925
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0225 - accuracy: 0.9925
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0227 - accuracy: 0.9924
1473/1688 [=========================>....] - ETA: 0s - loss: 0.0229 - accuracy: 0.9924
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0228 - accuracy: 0.9924
1518/1688 [=========================>....] - ETA: 0s - loss: 0.0229 - accuracy: 0.9924
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0228 - accuracy: 0.9924
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0230 - accuracy: 0.9922
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0231 - accuracy: 0.9923
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0232 - accuracy: 0.9922
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0232 - accuracy: 0.9922
1655/1688 [============================>.] - ETA: 0s - loss: 0.0233 - accuracy: 0.9922
1677/1688 [============================>.] - ETA: 0s - loss: 0.0232 - accuracy: 0.9921
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0231 - accuracy: 0.9921 - val_loss: 0.0524 - val_accuracy: 0.9862
Epoch 8/10
1/1688 [..............................] - ETA: 4s - loss: 5.4374e-05 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0194 - accuracy: 0.9932
45/1688 [..............................] - ETA: 3s - loss: 0.0221 - accuracy: 0.9917
67/1688 [>.............................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9939
90/1688 [>.............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9944
112/1688 [>.............................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9953
135/1688 [=>............................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9954
158/1688 [=>............................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9956
179/1688 [==>...........................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9960
200/1688 [==>...........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9962
222/1688 [==>...........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9959
245/1688 [===>..........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9963
267/1688 [===>..........................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9961
290/1688 [====>.........................] - ETA: 3s - loss: 0.0120 - accuracy: 0.9963
312/1688 [====>.........................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9966
335/1688 [====>.........................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9965
357/1688 [=====>........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9965
379/1688 [=====>........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9966
402/1688 [======>.......................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9965
424/1688 [======>.......................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9959
447/1688 [======>.......................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9957
469/1688 [=======>......................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9953
491/1688 [=======>......................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9951
513/1688 [========>.....................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9950
536/1688 [========>.....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9951
558/1688 [========>.....................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9950
580/1688 [=========>....................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9949
603/1688 [=========>....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9950
627/1688 [==========>...................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9950
650/1688 [==========>...................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9951
672/1688 [==========>...................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9950
694/1688 [===========>..................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9948
717/1688 [===========>..................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9948
739/1688 [============>.................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9947
761/1688 [============>.................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9946
783/1688 [============>.................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9944
806/1688 [=============>................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9943
829/1688 [=============>................] - ETA: 1s - loss: 0.0181 - accuracy: 0.9942
852/1688 [==============>...............] - ETA: 1s - loss: 0.0182 - accuracy: 0.9942
874/1688 [==============>...............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9940
896/1688 [==============>...............] - ETA: 1s - loss: 0.0183 - accuracy: 0.9940
919/1688 [===============>..............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9940
942/1688 [===============>..............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9941
964/1688 [================>.............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9941
986/1688 [================>.............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9940
1008/1688 [================>.............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9940
1030/1688 [=================>............] - ETA: 1s - loss: 0.0197 - accuracy: 0.9939
1053/1688 [=================>............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9939
1075/1688 [==================>...........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9937
1098/1688 [==================>...........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9937
1121/1688 [==================>...........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9936
1143/1688 [===================>..........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9936
1165/1688 [===================>..........] - ETA: 1s - loss: 0.0208 - accuracy: 0.9935
1188/1688 [====================>.........] - ETA: 1s - loss: 0.0210 - accuracy: 0.9935
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0211 - accuracy: 0.9934
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9933
1256/1688 [=====================>........] - ETA: 0s - loss: 0.0212 - accuracy: 0.9934
1279/1688 [=====================>........] - ETA: 0s - loss: 0.0213 - accuracy: 0.9933
1302/1688 [======================>.......] - ETA: 0s - loss: 0.0216 - accuracy: 0.9933
1325/1688 [======================>.......] - ETA: 0s - loss: 0.0215 - accuracy: 0.9933
1348/1688 [======================>.......] - ETA: 0s - loss: 0.0216 - accuracy: 0.9932
1371/1688 [=======================>......] - ETA: 0s - loss: 0.0215 - accuracy: 0.9932
1394/1688 [=======================>......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9932
1416/1688 [========================>.....] - ETA: 0s - loss: 0.0215 - accuracy: 0.9932
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0216 - accuracy: 0.9932
1462/1688 [========================>.....] - ETA: 0s - loss: 0.0216 - accuracy: 0.9932
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0217 - accuracy: 0.9931
1506/1688 [=========================>....] - ETA: 0s - loss: 0.0216 - accuracy: 0.9931
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0215 - accuracy: 0.9931
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0215 - accuracy: 0.9931
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0215 - accuracy: 0.9931
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0215 - accuracy: 0.9930
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0214 - accuracy: 0.9931
1640/1688 [============================>.] - ETA: 0s - loss: 0.0213 - accuracy: 0.9931
1663/1688 [============================>.] - ETA: 0s - loss: 0.0213 - accuracy: 0.9931
1687/1688 [============================>.] - ETA: 0s - loss: 0.0215 - accuracy: 0.9930
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0215 - accuracy: 0.9930 - val_loss: 0.0526 - val_accuracy: 0.9873
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 6.9804e-05 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9959
45/1688 [..............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9958
67/1688 [>.............................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9967
89/1688 [>.............................] - ETA: 3s - loss: 0.0117 - accuracy: 0.9961
111/1688 [>.............................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9966
133/1688 [=>............................] - ETA: 3s - loss: 0.0099 - accuracy: 0.9972
155/1688 [=>............................] - ETA: 3s - loss: 0.0100 - accuracy: 0.9972
177/1688 [==>...........................] - ETA: 3s - loss: 0.0104 - accuracy: 0.9970
199/1688 [==>...........................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9962
222/1688 [==>...........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9959
245/1688 [===>..........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9958
267/1688 [===>..........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9959
290/1688 [====>.........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9961
312/1688 [====>.........................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9962
334/1688 [====>.........................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9963
357/1688 [=====>........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9965
380/1688 [=====>........................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9965
402/1688 [======>.......................] - ETA: 2s - loss: 0.0117 - accuracy: 0.9966
424/1688 [======>.......................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9966
447/1688 [======>.......................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9964
470/1688 [=======>......................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9965
492/1688 [=======>......................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9965
514/1688 [========>.....................] - ETA: 2s - loss: 0.0112 - accuracy: 0.9965
537/1688 [========>.....................] - ETA: 2s - loss: 0.0111 - accuracy: 0.9966
559/1688 [========>.....................] - ETA: 2s - loss: 0.0110 - accuracy: 0.9966
582/1688 [=========>....................] - ETA: 2s - loss: 0.0109 - accuracy: 0.9967
605/1688 [=========>....................] - ETA: 2s - loss: 0.0111 - accuracy: 0.9967
627/1688 [==========>...................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9967
650/1688 [==========>...................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
673/1688 [==========>...................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9966
696/1688 [===========>..................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9966
718/1688 [===========>..................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9966
741/1688 [============>.................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9966
764/1688 [============>.................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9967
786/1688 [============>.................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9965
809/1688 [=============>................] - ETA: 1s - loss: 0.0116 - accuracy: 0.9965
832/1688 [=============>................] - ETA: 1s - loss: 0.0120 - accuracy: 0.9965
854/1688 [==============>...............] - ETA: 1s - loss: 0.0123 - accuracy: 0.9963
877/1688 [==============>...............] - ETA: 1s - loss: 0.0125 - accuracy: 0.9962
900/1688 [==============>...............] - ETA: 1s - loss: 0.0127 - accuracy: 0.9961
922/1688 [===============>..............] - ETA: 1s - loss: 0.0128 - accuracy: 0.9960
945/1688 [===============>..............] - ETA: 1s - loss: 0.0128 - accuracy: 0.9960
968/1688 [================>.............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9959
991/1688 [================>.............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9958
1013/1688 [=================>............] - ETA: 1s - loss: 0.0135 - accuracy: 0.9958
1035/1688 [=================>............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9957
1057/1688 [=================>............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9957
1079/1688 [==================>...........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9956
1101/1688 [==================>...........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9955
1124/1688 [==================>...........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9954
1147/1688 [===================>..........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9954
1169/1688 [===================>..........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9954
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9954
1213/1688 [====================>.........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9954
1235/1688 [====================>.........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9954
1256/1688 [=====================>........] - ETA: 0s - loss: 0.0139 - accuracy: 0.9954
1279/1688 [=====================>........] - ETA: 0s - loss: 0.0141 - accuracy: 0.9953
1301/1688 [======================>.......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9953
1323/1688 [======================>.......] - ETA: 0s - loss: 0.0144 - accuracy: 0.9952
1346/1688 [======================>.......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9952
1368/1688 [=======================>......] - ETA: 0s - loss: 0.0145 - accuracy: 0.9952
1390/1688 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9952
1412/1688 [========================>.....] - ETA: 0s - loss: 0.0147 - accuracy: 0.9952
1434/1688 [========================>.....] - ETA: 0s - loss: 0.0147 - accuracy: 0.9952
1457/1688 [========================>.....] - ETA: 0s - loss: 0.0148 - accuracy: 0.9952
1480/1688 [=========================>....] - ETA: 0s - loss: 0.0148 - accuracy: 0.9951
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0147 - accuracy: 0.9952
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0148 - accuracy: 0.9951
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0150 - accuracy: 0.9951
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0151 - accuracy: 0.9951
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0150 - accuracy: 0.9951
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0153 - accuracy: 0.9950
1637/1688 [============================>.] - ETA: 0s - loss: 0.0153 - accuracy: 0.9950
1660/1688 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9950
1683/1688 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9950
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0156 - accuracy: 0.9949 - val_loss: 0.0668 - val_accuracy: 0.9850
Epoch 10/10
1/1688 [..............................] - ETA: 4s - loss: 0.0119 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0115 - accuracy: 0.9946
46/1688 [..............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9932
68/1688 [>.............................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9936
90/1688 [>.............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9941
113/1688 [=>............................] - ETA: 3s - loss: 0.0120 - accuracy: 0.9950
135/1688 [=>............................] - ETA: 3s - loss: 0.0115 - accuracy: 0.9954
157/1688 [=>............................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9956
179/1688 [==>...........................] - ETA: 3s - loss: 0.0119 - accuracy: 0.9956
202/1688 [==>...........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9954
224/1688 [==>...........................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9953
247/1688 [===>..........................] - ETA: 3s - loss: 0.0120 - accuracy: 0.9953
270/1688 [===>..........................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9954
293/1688 [====>.........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9953
316/1688 [====>.........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9955
339/1688 [=====>........................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9954
361/1688 [=====>........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9955
383/1688 [=====>........................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9954
406/1688 [======>.......................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9953
429/1688 [======>.......................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9953
452/1688 [=======>......................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9954
474/1688 [=======>......................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9953
497/1688 [=======>......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9952
519/1688 [========>.....................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9952
541/1688 [========>.....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9951
563/1688 [=========>....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9950
585/1688 [=========>....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9949
608/1688 [=========>....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9949
630/1688 [==========>...................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9949
653/1688 [==========>...................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9949
675/1688 [==========>...................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9949
698/1688 [===========>..................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9948
722/1688 [===========>..................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9948
744/1688 [============>.................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9949
767/1688 [============>.................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9947
790/1688 [=============>................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9948
813/1688 [=============>................] - ETA: 1s - loss: 0.0151 - accuracy: 0.9947
837/1688 [=============>................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9948
860/1688 [==============>...............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9948
883/1688 [==============>...............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9948
906/1688 [===============>..............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9948
929/1688 [===============>..............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9949
951/1688 [===============>..............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9949
973/1688 [================>.............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9949
995/1688 [================>.............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9948
1017/1688 [=================>............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9948
1041/1688 [=================>............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9947
1065/1688 [=================>............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9947
1088/1688 [==================>...........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9947
1111/1688 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9947
1134/1688 [===================>..........] - ETA: 1s - loss: 0.0153 - accuracy: 0.9945
1157/1688 [===================>..........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9945
1179/1688 [===================>..........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9945
1201/1688 [====================>.........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9945
1224/1688 [====================>.........] - ETA: 1s - loss: 0.0156 - accuracy: 0.9943
1246/1688 [=====================>........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9943
1268/1688 [=====================>........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9943
1290/1688 [=====================>........] - ETA: 0s - loss: 0.0157 - accuracy: 0.9942
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9943
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9943
1359/1688 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9943
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9942
1404/1688 [=======================>......] - ETA: 0s - loss: 0.0157 - accuracy: 0.9942
1426/1688 [========================>.....] - ETA: 0s - loss: 0.0156 - accuracy: 0.9942
1449/1688 [========================>.....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9942
1472/1688 [=========================>....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9941
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0161 - accuracy: 0.9941
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0161 - accuracy: 0.9941
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0160 - accuracy: 0.9941
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0161 - accuracy: 0.9941
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0160 - accuracy: 0.9941
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0160 - accuracy: 0.9941
1632/1688 [============================>.] - ETA: 0s - loss: 0.0161 - accuracy: 0.9941
1654/1688 [============================>.] - ETA: 0s - loss: 0.0164 - accuracy: 0.9940
1676/1688 [============================>.] - ETA: 0s - loss: 0.0164 - accuracy: 0.9941
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0164 - accuracy: 0.9941 - val_loss: 0.0719 - val_accuracy: 0.9840
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9825999736785889
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bit integer activations and weights. Therefore, the floating-point TF-Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a TF-Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating-point TF-Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a TF-Keras model with custom QuantizeML quantized layers substituted for the original TF-Keras layers.
All TF-Keras API functions can be applied on this new model: summary(), compile(),
fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:577: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples. Continuing execution.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 2:32
51/1024 [>.............................] - ETA: 0s
103/1024 [==>...........................] - ETA: 0s
155/1024 [===>..........................] - ETA: 0s
206/1024 [=====>........................] - ETA: 0s
258/1024 [======>.......................] - ETA: 0s
310/1024 [========>.....................] - ETA: 0s
362/1024 [=========>....................] - ETA: 0s
415/1024 [===========>..................] - ETA: 0s
467/1024 [============>.................] - ETA: 0s
519/1024 [==============>...............] - ETA: 0s
571/1024 [===============>..............] - ETA: 0s
623/1024 [=================>............] - ETA: 0s
675/1024 [==================>...........] - ETA: 0s
727/1024 [====================>.........] - ETA: 0s
779/1024 [=====================>........] - ETA: 0s
831/1024 [=======================>......] - ETA: 0s
883/1024 [========================>.....] - ETA: 0s
935/1024 [==========================>...] - ETA: 0s
987/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 972us/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 28, 28, 1)] 0
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9735999703407288
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9807000160217285
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating-point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9778000116348267
2.4. Model fine-tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine-tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine-tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 1:48:01 - loss: 0.0110 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0308 - accuracy: 0.9792
17/1688 [..............................] - ETA: 10s - loss: 0.0316 - accuracy: 0.9816
25/1688 [..............................] - ETA: 10s - loss: 0.0419 - accuracy: 0.9812
33/1688 [..............................] - ETA: 10s - loss: 0.0388 - accuracy: 0.9830
41/1688 [..............................] - ETA: 10s - loss: 0.0365 - accuracy: 0.9855
49/1688 [..............................] - ETA: 10s - loss: 0.0333 - accuracy: 0.9866
57/1688 [>.............................] - ETA: 10s - loss: 0.0321 - accuracy: 0.9868
65/1688 [>.............................] - ETA: 10s - loss: 0.0301 - accuracy: 0.9880
74/1688 [>.............................] - ETA: 10s - loss: 0.0278 - accuracy: 0.9890
83/1688 [>.............................] - ETA: 10s - loss: 0.0269 - accuracy: 0.9898
92/1688 [>.............................] - ETA: 10s - loss: 0.0265 - accuracy: 0.9901
101/1688 [>.............................] - ETA: 10s - loss: 0.0262 - accuracy: 0.9904
110/1688 [>.............................] - ETA: 9s - loss: 0.0253 - accuracy: 0.9909
119/1688 [=>............................] - ETA: 9s - loss: 0.0238 - accuracy: 0.9916
128/1688 [=>............................] - ETA: 9s - loss: 0.0235 - accuracy: 0.9917
137/1688 [=>............................] - ETA: 9s - loss: 0.0229 - accuracy: 0.9918
146/1688 [=>............................] - ETA: 9s - loss: 0.0224 - accuracy: 0.9921
155/1688 [=>............................] - ETA: 9s - loss: 0.0220 - accuracy: 0.9923
164/1688 [=>............................] - ETA: 9s - loss: 0.0222 - accuracy: 0.9920
172/1688 [==>...........................] - ETA: 9s - loss: 0.0217 - accuracy: 0.9924
180/1688 [==>...........................] - ETA: 9s - loss: 0.0221 - accuracy: 0.9922
189/1688 [==>...........................] - ETA: 9s - loss: 0.0219 - accuracy: 0.9922
198/1688 [==>...........................] - ETA: 9s - loss: 0.0219 - accuracy: 0.9923
207/1688 [==>...........................] - ETA: 9s - loss: 0.0215 - accuracy: 0.9925
216/1688 [==>...........................] - ETA: 9s - loss: 0.0231 - accuracy: 0.9922
224/1688 [==>...........................] - ETA: 9s - loss: 0.0224 - accuracy: 0.9925
233/1688 [===>..........................] - ETA: 9s - loss: 0.0237 - accuracy: 0.9922
241/1688 [===>..........................] - ETA: 9s - loss: 0.0234 - accuracy: 0.9922
250/1688 [===>..........................] - ETA: 8s - loss: 0.0231 - accuracy: 0.9924
259/1688 [===>..........................] - ETA: 8s - loss: 0.0225 - accuracy: 0.9926
267/1688 [===>..........................] - ETA: 8s - loss: 0.0221 - accuracy: 0.9929
275/1688 [===>..........................] - ETA: 8s - loss: 0.0216 - accuracy: 0.9931
284/1688 [====>.........................] - ETA: 8s - loss: 0.0211 - accuracy: 0.9932
293/1688 [====>.........................] - ETA: 8s - loss: 0.0208 - accuracy: 0.9933
302/1688 [====>.........................] - ETA: 8s - loss: 0.0207 - accuracy: 0.9933
310/1688 [====>.........................] - ETA: 8s - loss: 0.0204 - accuracy: 0.9933
319/1688 [====>.........................] - ETA: 8s - loss: 0.0200 - accuracy: 0.9934
328/1688 [====>.........................] - ETA: 8s - loss: 0.0198 - accuracy: 0.9933
336/1688 [====>.........................] - ETA: 8s - loss: 0.0194 - accuracy: 0.9935
345/1688 [=====>........................] - ETA: 8s - loss: 0.0192 - accuracy: 0.9936
354/1688 [=====>........................] - ETA: 8s - loss: 0.0190 - accuracy: 0.9936
363/1688 [=====>........................] - ETA: 8s - loss: 0.0191 - accuracy: 0.9936
371/1688 [=====>........................] - ETA: 8s - loss: 0.0188 - accuracy: 0.9937
379/1688 [=====>........................] - ETA: 8s - loss: 0.0187 - accuracy: 0.9937
387/1688 [=====>........................] - ETA: 8s - loss: 0.0187 - accuracy: 0.9937
395/1688 [======>.......................] - ETA: 8s - loss: 0.0184 - accuracy: 0.9938
403/1688 [======>.......................] - ETA: 7s - loss: 0.0187 - accuracy: 0.9938
411/1688 [======>.......................] - ETA: 7s - loss: 0.0184 - accuracy: 0.9939
420/1688 [======>.......................] - ETA: 7s - loss: 0.0184 - accuracy: 0.9939
429/1688 [======>.......................] - ETA: 7s - loss: 0.0182 - accuracy: 0.9940
438/1688 [======>.......................] - ETA: 7s - loss: 0.0180 - accuracy: 0.9939
447/1688 [======>.......................] - ETA: 7s - loss: 0.0178 - accuracy: 0.9940
456/1688 [=======>......................] - ETA: 7s - loss: 0.0180 - accuracy: 0.9938
465/1688 [=======>......................] - ETA: 7s - loss: 0.0177 - accuracy: 0.9940
474/1688 [=======>......................] - ETA: 7s - loss: 0.0179 - accuracy: 0.9939
483/1688 [=======>......................] - ETA: 7s - loss: 0.0178 - accuracy: 0.9939
491/1688 [=======>......................] - ETA: 7s - loss: 0.0179 - accuracy: 0.9939
500/1688 [=======>......................] - ETA: 7s - loss: 0.0180 - accuracy: 0.9939
508/1688 [========>.....................] - ETA: 7s - loss: 0.0179 - accuracy: 0.9940
516/1688 [========>.....................] - ETA: 7s - loss: 0.0177 - accuracy: 0.9941
525/1688 [========>.....................] - ETA: 7s - loss: 0.0176 - accuracy: 0.9942
533/1688 [========>.....................] - ETA: 7s - loss: 0.0174 - accuracy: 0.9943
542/1688 [========>.....................] - ETA: 7s - loss: 0.0171 - accuracy: 0.9943
550/1688 [========>.....................] - ETA: 7s - loss: 0.0170 - accuracy: 0.9944
558/1688 [========>.....................] - ETA: 7s - loss: 0.0168 - accuracy: 0.9945
567/1688 [=========>....................] - ETA: 6s - loss: 0.0168 - accuracy: 0.9945
575/1688 [=========>....................] - ETA: 6s - loss: 0.0167 - accuracy: 0.9945
583/1688 [=========>....................] - ETA: 6s - loss: 0.0167 - accuracy: 0.9945
592/1688 [=========>....................] - ETA: 6s - loss: 0.0168 - accuracy: 0.9944
601/1688 [=========>....................] - ETA: 6s - loss: 0.0166 - accuracy: 0.9945
609/1688 [=========>....................] - ETA: 6s - loss: 0.0165 - accuracy: 0.9945
618/1688 [=========>....................] - ETA: 6s - loss: 0.0167 - accuracy: 0.9944
627/1688 [==========>...................] - ETA: 6s - loss: 0.0165 - accuracy: 0.9945
636/1688 [==========>...................] - ETA: 6s - loss: 0.0164 - accuracy: 0.9945
644/1688 [==========>...................] - ETA: 6s - loss: 0.0163 - accuracy: 0.9946
653/1688 [==========>...................] - ETA: 6s - loss: 0.0162 - accuracy: 0.9946
662/1688 [==========>...................] - ETA: 6s - loss: 0.0163 - accuracy: 0.9946
670/1688 [==========>...................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9946
679/1688 [===========>..................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9946
688/1688 [===========>..................] - ETA: 6s - loss: 0.0160 - accuracy: 0.9947
697/1688 [===========>..................] - ETA: 6s - loss: 0.0159 - accuracy: 0.9947
706/1688 [===========>..................] - ETA: 6s - loss: 0.0160 - accuracy: 0.9946
714/1688 [===========>..................] - ETA: 6s - loss: 0.0159 - accuracy: 0.9947
723/1688 [===========>..................] - ETA: 5s - loss: 0.0159 - accuracy: 0.9947
732/1688 [============>.................] - ETA: 5s - loss: 0.0158 - accuracy: 0.9947
741/1688 [============>.................] - ETA: 5s - loss: 0.0157 - accuracy: 0.9948
750/1688 [============>.................] - ETA: 5s - loss: 0.0156 - accuracy: 0.9948
758/1688 [============>.................] - ETA: 5s - loss: 0.0155 - accuracy: 0.9948
767/1688 [============>.................] - ETA: 5s - loss: 0.0156 - accuracy: 0.9949
775/1688 [============>.................] - ETA: 5s - loss: 0.0156 - accuracy: 0.9949
784/1688 [============>.................] - ETA: 5s - loss: 0.0154 - accuracy: 0.9949
793/1688 [=============>................] - ETA: 5s - loss: 0.0153 - accuracy: 0.9949
802/1688 [=============>................] - ETA: 5s - loss: 0.0153 - accuracy: 0.9949
810/1688 [=============>................] - ETA: 5s - loss: 0.0152 - accuracy: 0.9949
819/1688 [=============>................] - ETA: 5s - loss: 0.0151 - accuracy: 0.9950
827/1688 [=============>................] - ETA: 5s - loss: 0.0150 - accuracy: 0.9950
836/1688 [=============>................] - ETA: 5s - loss: 0.0150 - accuracy: 0.9950
845/1688 [==============>...............] - ETA: 5s - loss: 0.0150 - accuracy: 0.9950
854/1688 [==============>...............] - ETA: 5s - loss: 0.0149 - accuracy: 0.9950
863/1688 [==============>...............] - ETA: 5s - loss: 0.0147 - accuracy: 0.9951
871/1688 [==============>...............] - ETA: 5s - loss: 0.0147 - accuracy: 0.9950
880/1688 [==============>...............] - ETA: 5s - loss: 0.0146 - accuracy: 0.9951
888/1688 [==============>...............] - ETA: 4s - loss: 0.0145 - accuracy: 0.9951
896/1688 [==============>...............] - ETA: 4s - loss: 0.0144 - accuracy: 0.9952
905/1688 [===============>..............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9952
914/1688 [===============>..............] - ETA: 4s - loss: 0.0144 - accuracy: 0.9952
923/1688 [===============>..............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9952
932/1688 [===============>..............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9953
940/1688 [===============>..............] - ETA: 4s - loss: 0.0141 - accuracy: 0.9953
948/1688 [===============>..............] - ETA: 4s - loss: 0.0140 - accuracy: 0.9953
957/1688 [================>.............] - ETA: 4s - loss: 0.0141 - accuracy: 0.9953
966/1688 [================>.............] - ETA: 4s - loss: 0.0140 - accuracy: 0.9954
975/1688 [================>.............] - ETA: 4s - loss: 0.0140 - accuracy: 0.9954
984/1688 [================>.............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9954
993/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9954
1002/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9954
1011/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9955
1020/1688 [=================>............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9955
1029/1688 [=================>............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9955
1038/1688 [=================>............] - ETA: 4s - loss: 0.0136 - accuracy: 0.9955
1046/1688 [=================>............] - ETA: 3s - loss: 0.0136 - accuracy: 0.9955
1055/1688 [=================>............] - ETA: 3s - loss: 0.0135 - accuracy: 0.9956
1064/1688 [=================>............] - ETA: 3s - loss: 0.0136 - accuracy: 0.9956
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0135 - accuracy: 0.9956
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9956
1091/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9956
1100/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9957
1109/1688 [==================>...........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9957
1117/1688 [==================>...........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9957
1126/1688 [===================>..........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9957
1134/1688 [===================>..........] - ETA: 3s - loss: 0.0132 - accuracy: 0.9957
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0132 - accuracy: 0.9957
1151/1688 [===================>..........] - ETA: 3s - loss: 0.0131 - accuracy: 0.9958
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0131 - accuracy: 0.9957
1168/1688 [===================>..........] - ETA: 3s - loss: 0.0131 - accuracy: 0.9957
1177/1688 [===================>..........] - ETA: 3s - loss: 0.0132 - accuracy: 0.9958
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0131 - accuracy: 0.9958
1195/1688 [====================>.........] - ETA: 3s - loss: 0.0131 - accuracy: 0.9957
1203/1688 [====================>.........] - ETA: 3s - loss: 0.0131 - accuracy: 0.9957
1212/1688 [====================>.........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9958
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1257/1688 [=====================>........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0131 - accuracy: 0.9958
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0131 - accuracy: 0.9958
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0130 - accuracy: 0.9958
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0130 - accuracy: 0.9958
1300/1688 [======================>.......] - ETA: 2s - loss: 0.0130 - accuracy: 0.9958
1309/1688 [======================>.......] - ETA: 2s - loss: 0.0129 - accuracy: 0.9958
1318/1688 [======================>.......] - ETA: 2s - loss: 0.0129 - accuracy: 0.9959
1327/1688 [======================>.......] - ETA: 2s - loss: 0.0130 - accuracy: 0.9959
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0130 - accuracy: 0.9959
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0129 - accuracy: 0.9959
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0129 - accuracy: 0.9959
1362/1688 [=======================>......] - ETA: 2s - loss: 0.0129 - accuracy: 0.9959
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0129 - accuracy: 0.9959
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0129 - accuracy: 0.9959
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0130 - accuracy: 0.9959
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0129 - accuracy: 0.9959
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0130 - accuracy: 0.9959
1411/1688 [========================>.....] - ETA: 1s - loss: 0.0130 - accuracy: 0.9959
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0129 - accuracy: 0.9959
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0129 - accuracy: 0.9959
1437/1688 [========================>.....] - ETA: 1s - loss: 0.0129 - accuracy: 0.9959
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0128 - accuracy: 0.9959
1454/1688 [========================>.....] - ETA: 1s - loss: 0.0128 - accuracy: 0.9960
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0127 - accuracy: 0.9960
1471/1688 [=========================>....] - ETA: 1s - loss: 0.0127 - accuracy: 0.9960
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9960
1487/1688 [=========================>....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9960
1496/1688 [=========================>....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9960
1504/1688 [=========================>....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9961
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9961
1521/1688 [==========================>...] - ETA: 1s - loss: 0.0126 - accuracy: 0.9961
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0126 - accuracy: 0.9961
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0126 - accuracy: 0.9961
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9961
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9961
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0124 - accuracy: 0.9961
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0124 - accuracy: 0.9962
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9962
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9962
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9961
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0125 - accuracy: 0.9961
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0125 - accuracy: 0.9962
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9962
1632/1688 [============================>.] - ETA: 0s - loss: 0.0124 - accuracy: 0.9962
1641/1688 [============================>.] - ETA: 0s - loss: 0.0124 - accuracy: 0.9962
1649/1688 [============================>.] - ETA: 0s - loss: 0.0124 - accuracy: 0.9962
1658/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9962
1667/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9962
1676/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9963
1684/1688 [============================>.] - ETA: 0s - loss: 0.0122 - accuracy: 0.9963
1688/1688 [==============================] - 16s 7ms/step - loss: 0.0122 - accuracy: 0.9963 - val_loss: 0.0623 - val_accuracy: 0.9868
Epoch 2/5
1/1688 [..............................] - ETA: 11s - loss: 0.0018 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0191 - accuracy: 0.9906
18/1688 [..............................] - ETA: 10s - loss: 0.0154 - accuracy: 0.9931
27/1688 [..............................] - ETA: 10s - loss: 0.0146 - accuracy: 0.9942
36/1688 [..............................] - ETA: 10s - loss: 0.0136 - accuracy: 0.9948
45/1688 [..............................] - ETA: 10s - loss: 0.0121 - accuracy: 0.9958
54/1688 [..............................] - ETA: 10s - loss: 0.0106 - accuracy: 0.9965
62/1688 [>.............................] - ETA: 10s - loss: 0.0105 - accuracy: 0.9960
70/1688 [>.............................] - ETA: 10s - loss: 0.0104 - accuracy: 0.9960
79/1688 [>.............................] - ETA: 10s - loss: 0.0098 - accuracy: 0.9960
88/1688 [>.............................] - ETA: 10s - loss: 0.0091 - accuracy: 0.9964
97/1688 [>.............................] - ETA: 9s - loss: 0.0097 - accuracy: 0.9961
106/1688 [>.............................] - ETA: 9s - loss: 0.0102 - accuracy: 0.9956
114/1688 [=>............................] - ETA: 9s - loss: 0.0099 - accuracy: 0.9956
123/1688 [=>............................] - ETA: 9s - loss: 0.0102 - accuracy: 0.9954
132/1688 [=>............................] - ETA: 9s - loss: 0.0101 - accuracy: 0.9955
140/1688 [=>............................] - ETA: 9s - loss: 0.0098 - accuracy: 0.9958
148/1688 [=>............................] - ETA: 9s - loss: 0.0097 - accuracy: 0.9958
156/1688 [=>............................] - ETA: 9s - loss: 0.0095 - accuracy: 0.9960
165/1688 [=>............................] - ETA: 9s - loss: 0.0093 - accuracy: 0.9962
174/1688 [==>...........................] - ETA: 9s - loss: 0.0094 - accuracy: 0.9960
182/1688 [==>...........................] - ETA: 9s - loss: 0.0091 - accuracy: 0.9962
191/1688 [==>...........................] - ETA: 9s - loss: 0.0090 - accuracy: 0.9962
200/1688 [==>...........................] - ETA: 9s - loss: 0.0090 - accuracy: 0.9961
209/1688 [==>...........................] - ETA: 9s - loss: 0.0089 - accuracy: 0.9961
218/1688 [==>...........................] - ETA: 9s - loss: 0.0087 - accuracy: 0.9963
227/1688 [===>..........................] - ETA: 9s - loss: 0.0086 - accuracy: 0.9963
235/1688 [===>..........................] - ETA: 9s - loss: 0.0084 - accuracy: 0.9964
243/1688 [===>..........................] - ETA: 9s - loss: 0.0085 - accuracy: 0.9964
252/1688 [===>..........................] - ETA: 8s - loss: 0.0085 - accuracy: 0.9964
260/1688 [===>..........................] - ETA: 8s - loss: 0.0083 - accuracy: 0.9965
269/1688 [===>..........................] - ETA: 8s - loss: 0.0082 - accuracy: 0.9966
278/1688 [===>..........................] - ETA: 8s - loss: 0.0081 - accuracy: 0.9966
287/1688 [====>.........................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9967
296/1688 [====>.........................] - ETA: 8s - loss: 0.0080 - accuracy: 0.9966
305/1688 [====>.........................] - ETA: 8s - loss: 0.0083 - accuracy: 0.9966
314/1688 [====>.........................] - ETA: 8s - loss: 0.0082 - accuracy: 0.9967
322/1688 [====>.........................] - ETA: 8s - loss: 0.0080 - accuracy: 0.9968
330/1688 [====>.........................] - ETA: 8s - loss: 0.0080 - accuracy: 0.9969
338/1688 [=====>........................] - ETA: 8s - loss: 0.0084 - accuracy: 0.9967
347/1688 [=====>........................] - ETA: 8s - loss: 0.0084 - accuracy: 0.9967
356/1688 [=====>........................] - ETA: 8s - loss: 0.0083 - accuracy: 0.9968
365/1688 [=====>........................] - ETA: 8s - loss: 0.0082 - accuracy: 0.9967
373/1688 [=====>........................] - ETA: 8s - loss: 0.0083 - accuracy: 0.9967
381/1688 [=====>........................] - ETA: 8s - loss: 0.0084 - accuracy: 0.9967
389/1688 [=====>........................] - ETA: 8s - loss: 0.0083 - accuracy: 0.9968
398/1688 [======>.......................] - ETA: 8s - loss: 0.0085 - accuracy: 0.9967
407/1688 [======>.......................] - ETA: 7s - loss: 0.0084 - accuracy: 0.9968
416/1688 [======>.......................] - ETA: 7s - loss: 0.0083 - accuracy: 0.9968
425/1688 [======>.......................] - ETA: 7s - loss: 0.0082 - accuracy: 0.9969
433/1688 [======>.......................] - ETA: 7s - loss: 0.0082 - accuracy: 0.9970
442/1688 [======>.......................] - ETA: 7s - loss: 0.0081 - accuracy: 0.9970
451/1688 [=======>......................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9971
460/1688 [=======>......................] - ETA: 7s - loss: 0.0081 - accuracy: 0.9971
468/1688 [=======>......................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9971
476/1688 [=======>......................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9972
484/1688 [=======>......................] - ETA: 7s - loss: 0.0078 - accuracy: 0.9972
493/1688 [=======>......................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9972
502/1688 [=======>......................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9973
511/1688 [========>.....................] - ETA: 7s - loss: 0.0081 - accuracy: 0.9972
520/1688 [========>.....................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9973
529/1688 [========>.....................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9973
537/1688 [========>.....................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9974
545/1688 [========>.....................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9974
553/1688 [========>.....................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9974
562/1688 [========>.....................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9974
571/1688 [=========>....................] - ETA: 6s - loss: 0.0078 - accuracy: 0.9975
580/1688 [=========>....................] - ETA: 6s - loss: 0.0080 - accuracy: 0.9975
588/1688 [=========>....................] - ETA: 6s - loss: 0.0080 - accuracy: 0.9974
596/1688 [=========>....................] - ETA: 6s - loss: 0.0080 - accuracy: 0.9975
605/1688 [=========>....................] - ETA: 6s - loss: 0.0080 - accuracy: 0.9975
614/1688 [=========>....................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9975
623/1688 [==========>...................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9974
632/1688 [==========>...................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9975
641/1688 [==========>...................] - ETA: 6s - loss: 0.0081 - accuracy: 0.9974
650/1688 [==========>...................] - ETA: 6s - loss: 0.0082 - accuracy: 0.9974
658/1688 [==========>...................] - ETA: 6s - loss: 0.0084 - accuracy: 0.9973
666/1688 [==========>...................] - ETA: 6s - loss: 0.0083 - accuracy: 0.9973
675/1688 [==========>...................] - ETA: 6s - loss: 0.0083 - accuracy: 0.9974
684/1688 [===========>..................] - ETA: 6s - loss: 0.0082 - accuracy: 0.9974
693/1688 [===========>..................] - ETA: 6s - loss: 0.0082 - accuracy: 0.9974
702/1688 [===========>..................] - ETA: 6s - loss: 0.0081 - accuracy: 0.9975
711/1688 [===========>..................] - ETA: 6s - loss: 0.0081 - accuracy: 0.9974
720/1688 [===========>..................] - ETA: 6s - loss: 0.0081 - accuracy: 0.9974
729/1688 [===========>..................] - ETA: 5s - loss: 0.0081 - accuracy: 0.9974
738/1688 [============>.................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9975
746/1688 [============>.................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9975
755/1688 [============>.................] - ETA: 5s - loss: 0.0079 - accuracy: 0.9975
764/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9975
772/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9975
781/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9976
790/1688 [=============>................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9976
798/1688 [=============>................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9975
807/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9976
816/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9976
824/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9976
833/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9976
841/1688 [=============>................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9975
850/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9976
858/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9975
866/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9975
875/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9975
884/1688 [==============>...............] - ETA: 4s - loss: 0.0080 - accuracy: 0.9975
893/1688 [==============>...............] - ETA: 4s - loss: 0.0081 - accuracy: 0.9975
902/1688 [===============>..............] - ETA: 4s - loss: 0.0081 - accuracy: 0.9975
911/1688 [===============>..............] - ETA: 4s - loss: 0.0083 - accuracy: 0.9974
920/1688 [===============>..............] - ETA: 4s - loss: 0.0082 - accuracy: 0.9975
929/1688 [===============>..............] - ETA: 4s - loss: 0.0082 - accuracy: 0.9975
937/1688 [===============>..............] - ETA: 4s - loss: 0.0082 - accuracy: 0.9974
946/1688 [===============>..............] - ETA: 4s - loss: 0.0082 - accuracy: 0.9975
955/1688 [===============>..............] - ETA: 4s - loss: 0.0083 - accuracy: 0.9974
963/1688 [================>.............] - ETA: 4s - loss: 0.0083 - accuracy: 0.9974
971/1688 [================>.............] - ETA: 4s - loss: 0.0083 - accuracy: 0.9974
979/1688 [================>.............] - ETA: 4s - loss: 0.0083 - accuracy: 0.9974
988/1688 [================>.............] - ETA: 4s - loss: 0.0083 - accuracy: 0.9974
997/1688 [================>.............] - ETA: 4s - loss: 0.0083 - accuracy: 0.9974
1006/1688 [================>.............] - ETA: 4s - loss: 0.0082 - accuracy: 0.9975
1014/1688 [=================>............] - ETA: 4s - loss: 0.0082 - accuracy: 0.9975
1023/1688 [=================>............] - ETA: 4s - loss: 0.0082 - accuracy: 0.9975
1032/1688 [=================>............] - ETA: 4s - loss: 0.0081 - accuracy: 0.9975
1041/1688 [=================>............] - ETA: 4s - loss: 0.0081 - accuracy: 0.9975
1049/1688 [=================>............] - ETA: 3s - loss: 0.0081 - accuracy: 0.9976
1058/1688 [=================>............] - ETA: 3s - loss: 0.0081 - accuracy: 0.9975
1066/1688 [=================>............] - ETA: 3s - loss: 0.0081 - accuracy: 0.9976
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0080 - accuracy: 0.9976
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0080 - accuracy: 0.9976
1091/1688 [==================>...........] - ETA: 3s - loss: 0.0080 - accuracy: 0.9976
1100/1688 [==================>...........] - ETA: 3s - loss: 0.0080 - accuracy: 0.9976
1109/1688 [==================>...........] - ETA: 3s - loss: 0.0080 - accuracy: 0.9976
1118/1688 [==================>...........] - ETA: 3s - loss: 0.0079 - accuracy: 0.9976
1126/1688 [===================>..........] - ETA: 3s - loss: 0.0079 - accuracy: 0.9976
1135/1688 [===================>..........] - ETA: 3s - loss: 0.0079 - accuracy: 0.9977
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0079 - accuracy: 0.9976
1151/1688 [===================>..........] - ETA: 3s - loss: 0.0078 - accuracy: 0.9977
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0078 - accuracy: 0.9977
1169/1688 [===================>..........] - ETA: 3s - loss: 0.0078 - accuracy: 0.9977
1178/1688 [===================>..........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9977
1187/1688 [====================>.........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9977
1196/1688 [====================>.........] - ETA: 3s - loss: 0.0078 - accuracy: 0.9977
1204/1688 [====================>.........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9977
1212/1688 [====================>.........] - ETA: 2s - loss: 0.0077 - accuracy: 0.9977
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0077 - accuracy: 0.9977
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9977
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9976
1247/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9976
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9976
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9976
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9975
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9976
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9976
1298/1688 [======================>.......] - ETA: 2s - loss: 0.0079 - accuracy: 0.9976
1306/1688 [======================>.......] - ETA: 2s - loss: 0.0079 - accuracy: 0.9976
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0078 - accuracy: 0.9976
1322/1688 [======================>.......] - ETA: 2s - loss: 0.0078 - accuracy: 0.9976
1331/1688 [======================>.......] - ETA: 2s - loss: 0.0078 - accuracy: 0.9976
1339/1688 [======================>.......] - ETA: 2s - loss: 0.0078 - accuracy: 0.9976
1348/1688 [======================>.......] - ETA: 2s - loss: 0.0078 - accuracy: 0.9976
1357/1688 [=======================>......] - ETA: 2s - loss: 0.0078 - accuracy: 0.9976
1366/1688 [=======================>......] - ETA: 1s - loss: 0.0078 - accuracy: 0.9976
1374/1688 [=======================>......] - ETA: 1s - loss: 0.0078 - accuracy: 0.9976
1382/1688 [=======================>......] - ETA: 1s - loss: 0.0078 - accuracy: 0.9976
1390/1688 [=======================>......] - ETA: 1s - loss: 0.0078 - accuracy: 0.9976
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1407/1688 [========================>.....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1416/1688 [========================>.....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0078 - accuracy: 0.9976
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1483/1688 [=========================>....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1491/1688 [=========================>....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1500/1688 [=========================>....] - ETA: 1s - loss: 0.0077 - accuracy: 0.9976
1509/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9977
1517/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9977
1526/1688 [==========================>...] - ETA: 1s - loss: 0.0076 - accuracy: 0.9977
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0077 - accuracy: 0.9977
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1639/1688 [============================>.] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1648/1688 [============================>.] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1657/1688 [============================>.] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1666/1688 [============================>.] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1675/1688 [============================>.] - ETA: 0s - loss: 0.0076 - accuracy: 0.9977
1684/1688 [============================>.] - ETA: 0s - loss: 0.0075 - accuracy: 0.9977
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0075 - accuracy: 0.9977 - val_loss: 0.0536 - val_accuracy: 0.9878
Epoch 3/5
1/1688 [..............................] - ETA: 12s - loss: 1.5629e-04 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0075 - accuracy: 0.9965
18/1688 [..............................] - ETA: 10s - loss: 0.0052 - accuracy: 0.9983
27/1688 [..............................] - ETA: 10s - loss: 0.0071 - accuracy: 0.9977
36/1688 [..............................] - ETA: 10s - loss: 0.0063 - accuracy: 0.9983
45/1688 [..............................] - ETA: 10s - loss: 0.0057 - accuracy: 0.9986
53/1688 [..............................] - ETA: 10s - loss: 0.0051 - accuracy: 0.9988
62/1688 [>.............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9990
71/1688 [>.............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9987
80/1688 [>.............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9988
89/1688 [>.............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9989
97/1688 [>.............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9990
105/1688 [>.............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9991
114/1688 [=>............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9989
123/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9990
131/1688 [=>............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9990
139/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9989
147/1688 [=>............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9987
156/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9988
164/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9989
172/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9989
180/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9990
189/1688 [==>...........................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9990
198/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9991
207/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9989
215/1688 [==>...........................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9990
224/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9989
233/1688 [===>..........................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9989
241/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9988
249/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9989
258/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9989
266/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9989
275/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9990
284/1688 [====>.........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9990
293/1688 [====>.........................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9989
301/1688 [====>.........................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9990
310/1688 [====>.........................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9990
319/1688 [====>.........................] - ETA: 8s - loss: 0.0051 - accuracy: 0.9989
327/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9989
335/1688 [====>.........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9988
344/1688 [=====>........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9988
353/1688 [=====>........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9988
361/1688 [=====>........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9988
370/1688 [=====>........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9986
379/1688 [=====>........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9985
388/1688 [=====>........................] - ETA: 8s - loss: 0.0064 - accuracy: 0.9982
396/1688 [======>.......................] - ETA: 8s - loss: 0.0063 - accuracy: 0.9983
404/1688 [======>.......................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9983
413/1688 [======>.......................] - ETA: 7s - loss: 0.0063 - accuracy: 0.9983
421/1688 [======>.......................] - ETA: 7s - loss: 0.0066 - accuracy: 0.9981
429/1688 [======>.......................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9982
438/1688 [======>.......................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9982
446/1688 [======>.......................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9982
455/1688 [=======>......................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9983
463/1688 [=======>......................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9983
472/1688 [=======>......................] - ETA: 7s - loss: 0.0063 - accuracy: 0.9983
481/1688 [=======>......................] - ETA: 7s - loss: 0.0066 - accuracy: 0.9983
489/1688 [=======>......................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9983
497/1688 [=======>......................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9984
505/1688 [=======>......................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9984
514/1688 [========>.....................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9984
523/1688 [========>.....................] - ETA: 7s - loss: 0.0063 - accuracy: 0.9984
532/1688 [========>.....................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9984
540/1688 [========>.....................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9984
549/1688 [========>.....................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9983
558/1688 [========>.....................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9984
567/1688 [=========>....................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
575/1688 [=========>....................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
584/1688 [=========>....................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9984
593/1688 [=========>....................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9985
602/1688 [=========>....................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
610/1688 [=========>....................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
618/1688 [=========>....................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9984
626/1688 [==========>...................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
635/1688 [==========>...................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
643/1688 [==========>...................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
652/1688 [==========>...................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9985
660/1688 [==========>...................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9985
668/1688 [==========>...................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9985
676/1688 [===========>..................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9985
685/1688 [===========>..................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9985
694/1688 [===========>..................] - ETA: 6s - loss: 0.0062 - accuracy: 0.9985
702/1688 [===========>..................] - ETA: 6s - loss: 0.0062 - accuracy: 0.9985
711/1688 [===========>..................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9985
720/1688 [===========>..................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9986
729/1688 [===========>..................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9985
738/1688 [============>.................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9985
746/1688 [============>.................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9984
755/1688 [============>.................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9984
764/1688 [============>.................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9984
773/1688 [============>.................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9985
782/1688 [============>.................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9985
791/1688 [=============>................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9985
800/1688 [=============>................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9985
808/1688 [=============>................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9985
817/1688 [=============>................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9985
825/1688 [=============>................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9984
833/1688 [=============>................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9985
842/1688 [=============>................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9984
850/1688 [==============>...............] - ETA: 5s - loss: 0.0064 - accuracy: 0.9985
859/1688 [==============>...............] - ETA: 5s - loss: 0.0063 - accuracy: 0.9985
868/1688 [==============>...............] - ETA: 5s - loss: 0.0063 - accuracy: 0.9985
877/1688 [==============>...............] - ETA: 5s - loss: 0.0063 - accuracy: 0.9985
886/1688 [==============>...............] - ETA: 4s - loss: 0.0064 - accuracy: 0.9984
895/1688 [==============>...............] - ETA: 4s - loss: 0.0064 - accuracy: 0.9985
903/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9984
912/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9985
921/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9984
930/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9985
938/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9985
947/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9984
955/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9984
964/1688 [================>.............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9984
973/1688 [================>.............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9984
981/1688 [================>.............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9984
990/1688 [================>.............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9984
999/1688 [================>.............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9984
1008/1688 [================>.............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9984
1016/1688 [=================>............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9985
1024/1688 [=================>............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9985
1033/1688 [=================>............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9985
1042/1688 [=================>............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9984
1051/1688 [=================>............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1059/1688 [=================>............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
1067/1688 [=================>............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
1075/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1101/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1110/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1118/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1126/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1135/1688 [===================>..........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9985
1152/1688 [===================>..........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1161/1688 [===================>..........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1170/1688 [===================>..........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1178/1688 [===================>..........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0060 - accuracy: 0.9986
1195/1688 [====================>.........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1204/1688 [====================>.........] - ETA: 3s - loss: 0.0060 - accuracy: 0.9985
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9986
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1257/1688 [=====================>........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9985
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9985
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9986
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9986
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9986
1299/1688 [======================>.......] - ETA: 2s - loss: 0.0060 - accuracy: 0.9986
1308/1688 [======================>.......] - ETA: 2s - loss: 0.0060 - accuracy: 0.9986
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0060 - accuracy: 0.9986
1326/1688 [======================>.......] - ETA: 2s - loss: 0.0059 - accuracy: 0.9986
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0059 - accuracy: 0.9986
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0059 - accuracy: 0.9986
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0059 - accuracy: 0.9986
1362/1688 [=======================>......] - ETA: 2s - loss: 0.0059 - accuracy: 0.9986
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1386/1688 [=======================>......] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1437/1688 [========================>.....] - ETA: 1s - loss: 0.0059 - accuracy: 0.9986
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1489/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1498/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1506/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9987
1515/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9987
1523/1688 [==========================>...] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9987
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1640/1688 [============================>.] - ETA: 0s - loss: 0.0057 - accuracy: 0.9986
1648/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1656/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1664/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1672/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1680/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0058 - accuracy: 0.9986 - val_loss: 0.0537 - val_accuracy: 0.9873
Epoch 4/5
1/1688 [..............................] - ETA: 12s - loss: 8.4068e-04 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0023 - accuracy: 1.0000
17/1688 [..............................] - ETA: 10s - loss: 0.0021 - accuracy: 1.0000
26/1688 [..............................] - ETA: 10s - loss: 0.0048 - accuracy: 0.9988
35/1688 [..............................] - ETA: 10s - loss: 0.0052 - accuracy: 0.9982
44/1688 [..............................] - ETA: 10s - loss: 0.0047 - accuracy: 0.9986
52/1688 [..............................] - ETA: 10s - loss: 0.0044 - accuracy: 0.9988
61/1688 [>.............................] - ETA: 10s - loss: 0.0044 - accuracy: 0.9990
70/1688 [>.............................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9987
79/1688 [>.............................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9988
88/1688 [>.............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9989
96/1688 [>.............................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9987
105/1688 [>.............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9988
114/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9989
123/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9990
131/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9988
140/1688 [=>............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9987
149/1688 [=>............................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9987
157/1688 [=>............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9988
165/1688 [=>............................] - ETA: 9s - loss: 0.0056 - accuracy: 0.9987
174/1688 [==>...........................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9987
183/1688 [==>...........................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9988
191/1688 [==>...........................] - ETA: 9s - loss: 0.0056 - accuracy: 0.9987
199/1688 [==>...........................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9987
207/1688 [==>...........................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9988
215/1688 [==>...........................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9988
224/1688 [==>...........................] - ETA: 9s - loss: 0.0056 - accuracy: 0.9987
233/1688 [===>..........................] - ETA: 9s - loss: 0.0057 - accuracy: 0.9987
242/1688 [===>..........................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9986
251/1688 [===>..........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9986
259/1688 [===>..........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9987
267/1688 [===>..........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9985
275/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9985
284/1688 [====>.........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9986
293/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9986
302/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9987
311/1688 [====>.........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9987
320/1688 [====>.........................] - ETA: 8s - loss: 0.0052 - accuracy: 0.9987
329/1688 [====>.........................] - ETA: 8s - loss: 0.0052 - accuracy: 0.9988
337/1688 [====>.........................] - ETA: 8s - loss: 0.0051 - accuracy: 0.9988
346/1688 [=====>........................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9988
355/1688 [=====>........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9989
363/1688 [=====>........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9989
372/1688 [=====>........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9989
380/1688 [=====>........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9988
388/1688 [=====>........................] - ETA: 8s - loss: 0.0052 - accuracy: 0.9988
396/1688 [======>.......................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9988
405/1688 [======>.......................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
414/1688 [======>.......................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
422/1688 [======>.......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
430/1688 [======>.......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
438/1688 [======>.......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
446/1688 [======>.......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
455/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
463/1688 [=======>......................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9987
472/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
481/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9986
489/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
498/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
507/1688 [========>.....................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9986
516/1688 [========>.....................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
524/1688 [========>.....................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
532/1688 [========>.....................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
540/1688 [========>.....................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9987
548/1688 [========>.....................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9987
557/1688 [========>.....................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9987
566/1688 [=========>....................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
575/1688 [=========>....................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
583/1688 [=========>....................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
591/1688 [=========>....................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9987
599/1688 [=========>....................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9987
608/1688 [=========>....................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
617/1688 [=========>....................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
626/1688 [==========>...................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9987
635/1688 [==========>...................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9987
644/1688 [==========>...................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9986
653/1688 [==========>...................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
662/1688 [==========>...................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
670/1688 [==========>...................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
678/1688 [===========>..................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
687/1688 [===========>..................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
695/1688 [===========>..................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
703/1688 [===========>..................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9988
711/1688 [===========>..................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
719/1688 [===========>..................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
728/1688 [===========>..................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
737/1688 [============>.................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
745/1688 [============>.................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
754/1688 [============>.................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
762/1688 [============>.................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
770/1688 [============>.................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
778/1688 [============>.................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
787/1688 [============>.................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
796/1688 [=============>................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
805/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
814/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
822/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
830/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
839/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
847/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
856/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
864/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
873/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
882/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
891/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
899/1688 [==============>...............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
908/1688 [===============>..............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
916/1688 [===============>..............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
925/1688 [===============>..............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
934/1688 [===============>..............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
942/1688 [===============>..............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
950/1688 [===============>..............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
959/1688 [================>.............] - ETA: 4s - loss: 0.0052 - accuracy: 0.9987
968/1688 [================>.............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
977/1688 [================>.............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
985/1688 [================>.............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
994/1688 [================>.............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
1002/1688 [================>.............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
1010/1688 [================>.............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
1019/1688 [=================>............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
1028/1688 [=================>............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
1036/1688 [=================>............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
1045/1688 [=================>............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9987
1054/1688 [=================>............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1063/1688 [=================>............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0051 - accuracy: 0.9987
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1098/1688 [==================>...........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1107/1688 [==================>...........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9988
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1124/1688 [==================>...........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1133/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1142/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1151/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1169/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1178/1688 [===================>..........] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1194/1688 [====================>.........] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1203/1688 [====================>.........] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1212/1688 [====================>.........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1257/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9988
1282/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1299/1688 [======================>.......] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1307/1688 [======================>.......] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1316/1688 [======================>.......] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1324/1688 [======================>.......] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1333/1688 [======================>.......] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1342/1688 [======================>.......] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1350/1688 [======================>.......] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9988
1366/1688 [=======================>......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9988
1374/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1427/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1469/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1487/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1495/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1503/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1511/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1520/1688 [==========================>...] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1639/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1648/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1656/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1665/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1674/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1683/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0050 - accuracy: 0.9989 - val_loss: 0.0582 - val_accuracy: 0.9868
Epoch 5/5
1/1688 [..............................] - ETA: 11s - loss: 8.9520e-04 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0013 - accuracy: 1.0000
17/1688 [..............................] - ETA: 10s - loss: 0.0018 - accuracy: 1.0000
25/1688 [..............................] - ETA: 10s - loss: 0.0017 - accuracy: 1.0000
33/1688 [..............................] - ETA: 10s - loss: 0.0020 - accuracy: 1.0000
41/1688 [..............................] - ETA: 10s - loss: 0.0019 - accuracy: 1.0000
50/1688 [..............................] - ETA: 10s - loss: 0.0018 - accuracy: 1.0000
59/1688 [>.............................] - ETA: 10s - loss: 0.0017 - accuracy: 1.0000
68/1688 [>.............................] - ETA: 10s - loss: 0.0018 - accuracy: 1.0000
76/1688 [>.............................] - ETA: 10s - loss: 0.0019 - accuracy: 1.0000
85/1688 [>.............................] - ETA: 10s - loss: 0.0025 - accuracy: 0.9996
93/1688 [>.............................] - ETA: 10s - loss: 0.0025 - accuracy: 0.9997
101/1688 [>.............................] - ETA: 9s - loss: 0.0026 - accuracy: 0.9997
110/1688 [>.............................] - ETA: 9s - loss: 0.0028 - accuracy: 0.9997
119/1688 [=>............................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9995
128/1688 [=>............................] - ETA: 9s - loss: 0.0029 - accuracy: 0.9995
137/1688 [=>............................] - ETA: 9s - loss: 0.0029 - accuracy: 0.9995
145/1688 [=>............................] - ETA: 9s - loss: 0.0029 - accuracy: 0.9996
154/1688 [=>............................] - ETA: 9s - loss: 0.0029 - accuracy: 0.9996
163/1688 [=>............................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9994
171/1688 [==>...........................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9995
180/1688 [==>...........................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9995
188/1688 [==>...........................] - ETA: 9s - loss: 0.0033 - accuracy: 0.9995
196/1688 [==>...........................] - ETA: 9s - loss: 0.0033 - accuracy: 0.9995
204/1688 [==>...........................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9994
213/1688 [==>...........................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9993
222/1688 [==>...........................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9993
231/1688 [===>..........................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9993
240/1688 [===>..........................] - ETA: 9s - loss: 0.0042 - accuracy: 0.9992
249/1688 [===>..........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9992
257/1688 [===>..........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9993
265/1688 [===>..........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9993
274/1688 [===>..........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9993
283/1688 [====>.........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9993
291/1688 [====>.........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9994
299/1688 [====>.........................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9994
307/1688 [====>.........................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9994
315/1688 [====>.........................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9994
323/1688 [====>.........................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9994
331/1688 [====>.........................] - ETA: 8s - loss: 0.0038 - accuracy: 0.9994
340/1688 [=====>........................] - ETA: 8s - loss: 0.0038 - accuracy: 0.9994
348/1688 [=====>........................] - ETA: 8s - loss: 0.0038 - accuracy: 0.9995
356/1688 [=====>........................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9995
365/1688 [=====>........................] - ETA: 8s - loss: 0.0038 - accuracy: 0.9995
373/1688 [=====>........................] - ETA: 8s - loss: 0.0038 - accuracy: 0.9995
382/1688 [=====>........................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9995
391/1688 [=====>........................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9995
400/1688 [======>.......................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9995
409/1688 [======>.......................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9995
418/1688 [======>.......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9995
426/1688 [======>.......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
435/1688 [======>.......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
444/1688 [======>.......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
453/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
462/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9995
471/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9995
480/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
489/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
497/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
506/1688 [=======>......................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9994
515/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9994
524/1688 [========>.....................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9994
532/1688 [========>.....................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9994
541/1688 [========>.....................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9994
550/1688 [========>.....................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9994
559/1688 [========>.....................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9993
568/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
576/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
584/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
593/1688 [=========>....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9993
601/1688 [=========>....................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
610/1688 [=========>....................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
618/1688 [=========>....................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
627/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
636/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
644/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
653/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
662/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
671/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
680/1688 [===========>..................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9991
688/1688 [===========>..................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
697/1688 [===========>..................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
706/1688 [===========>..................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9992
714/1688 [===========>..................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9992
723/1688 [===========>..................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
731/1688 [===========>..................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9991
739/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
748/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
757/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
765/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
773/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
782/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
790/1688 [=============>................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
798/1688 [=============>................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
807/1688 [=============>................] - ETA: 5s - loss: 0.0043 - accuracy: 0.9992
816/1688 [=============>................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
824/1688 [=============>................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
833/1688 [=============>................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
842/1688 [=============>................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
851/1688 [==============>...............] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
859/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
867/1688 [==============>...............] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
876/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
884/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
893/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
902/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
911/1688 [===============>..............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9991
920/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
929/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
937/1688 [===============>..............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9991
946/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
954/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
962/1688 [================>.............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9991
970/1688 [================>.............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9991
979/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
988/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
997/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
1006/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
1014/1688 [=================>............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
1023/1688 [=================>............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
1032/1688 [=================>............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
1040/1688 [=================>............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
1049/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1057/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1066/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1075/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1100/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1108/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1117/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1125/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1133/1688 [===================>..........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9991
1142/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1151/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9990
1159/1688 [===================>..........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9991
1167/1688 [===================>..........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9991
1176/1688 [===================>..........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9991
1184/1688 [====================>.........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9991
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9991
1201/1688 [====================>.........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9991
1209/1688 [====================>.........] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1312/1688 [======================>.......] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1320/1688 [======================>.......] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1329/1688 [======================>.......] - ETA: 2s - loss: 0.0045 - accuracy: 0.9991
1337/1688 [======================>.......] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1346/1688 [======================>.......] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1354/1688 [=======================>......] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1362/1688 [=======================>......] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1406/1688 [=======================>......] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9992
1450/1688 [========================>.....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9992
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1467/1688 [=========================>....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1483/1688 [=========================>....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1491/1688 [=========================>....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1500/1688 [=========================>....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1508/1688 [=========================>....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1517/1688 [=========================>....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1526/1688 [==========================>...] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9992
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9992
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9992
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0043 - accuracy: 0.9992
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0043 - accuracy: 0.9992
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0043 - accuracy: 0.9992
1601/1688 [===========================>..] - ETA: 0s - loss: 0.0043 - accuracy: 0.9992
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0043 - accuracy: 0.9992
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1635/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1644/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1653/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1662/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1670/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1678/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1687/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0044 - accuracy: 0.9991 - val_loss: 0.0566 - val_accuracy: 0.9877
<tf_keras.src.callbacks.History object at 0x779c764daed0>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine-tuning:', score)
Test accuracy after fine-tuning: 0.987500011920929
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with TF-Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test.astype(np.int32))
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9850000143051147
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-13.319714 -10.462454 -6.0926223 1.1227406 -15.733521 -7.5743504
-27.2958 10.708221 -7.467607 -0.2367952]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 22.198 seconds)