Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a TF-Keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original TF-Keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf_keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import tensorflow as tf
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from tf_keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, height, width, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
16384/11490434 [..............................] - ETA: 45s
49152/11490434 [..............................] - ETA: 36s
81920/11490434 [..............................] - ETA: 34s
147456/11490434 [..............................] - ETA: 23s
212992/11490434 [..............................] - ETA: 19s
327680/11490434 [..............................] - ETA: 14s
458752/11490434 [>.............................] - ETA: 11s
704512/11490434 [>.............................] - ETA: 7s
1114112/11490434 [=>............................] - ETA: 5s
1712128/11490434 [===>..........................] - ETA: 3s
1859584/11490434 [===>..........................] - ETA: 3s
1957888/11490434 [====>.........................] - ETA: 3s
2809856/11490434 [======>.......................] - ETA: 2s
3424256/11490434 [=======>......................] - ETA: 1s
3932160/11490434 [=========>....................] - ETA: 1s
4767744/11490434 [===========>..................] - ETA: 1s
5120000/11490434 [============>.................] - ETA: 1s
5816320/11490434 [==============>...............] - ETA: 1s
6356992/11490434 [===============>..............] - ETA: 0s
7135232/11490434 [=================>............] - ETA: 0s
7446528/11490434 [==================>...........] - ETA: 0s
7708672/11490434 [===================>..........] - ETA: 0s
8609792/11490434 [=====================>........] - ETA: 0s
8822784/11490434 [======================>.......] - ETA: 0s
9601024/11490434 [========================>.....] - ETA: 0s
9846784/11490434 [========================>.....] - ETA: 0s
9945088/11490434 [========================>.....] - ETA: 0s
10043392/11490434 [=========================>....] - ETA: 0s
10371072/11490434 [==========================>...] - ETA: 0s
10502144/11490434 [==========================>...] - ETA: 0s
10567680/11490434 [==========================>...] - ETA: 0s
10698752/11490434 [==========================>...] - ETA: 0s
10862592/11490434 [===========================>..] - ETA: 0s
10960896/11490434 [===========================>..] - ETA: 0s
11255808/11490434 [============================>.] - ETA: 0s
11452416/11490434 [============================>.] - ETA: 0s
11490434/11490434 [==============================] - 2s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard TF-Keras CNN model.
import tf_keras as keras
model_keras = keras.models.Sequential([
keras.layers.Input(shape=(28, 28, 1), name="input", dtype=tf.uint8),
keras.layers.Rescaling(1. / 255),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from tf_keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:01:29 - loss: 2.8031 - accuracy: 0.0625
22/1688 [..............................] - ETA: 4s - loss: 1.4597 - accuracy: 0.5284
44/1688 [..............................] - ETA: 3s - loss: 1.0602 - accuracy: 0.6683
66/1688 [>.............................] - ETA: 3s - loss: 0.8716 - accuracy: 0.7320
89/1688 [>.............................] - ETA: 3s - loss: 0.7614 - accuracy: 0.7658
111/1688 [>.............................] - ETA: 3s - loss: 0.6816 - accuracy: 0.7894
133/1688 [=>............................] - ETA: 3s - loss: 0.6275 - accuracy: 0.8050
156/1688 [=>............................] - ETA: 3s - loss: 0.5845 - accuracy: 0.8185
178/1688 [==>...........................] - ETA: 3s - loss: 0.5541 - accuracy: 0.8274
201/1688 [==>...........................] - ETA: 3s - loss: 0.5245 - accuracy: 0.8366
223/1688 [==>...........................] - ETA: 3s - loss: 0.4971 - accuracy: 0.8447
245/1688 [===>..........................] - ETA: 3s - loss: 0.4782 - accuracy: 0.8515
266/1688 [===>..........................] - ETA: 3s - loss: 0.4657 - accuracy: 0.8563
288/1688 [====>.........................] - ETA: 3s - loss: 0.4464 - accuracy: 0.8623
310/1688 [====>.........................] - ETA: 3s - loss: 0.4348 - accuracy: 0.8667
331/1688 [====>.........................] - ETA: 3s - loss: 0.4188 - accuracy: 0.8720
353/1688 [=====>........................] - ETA: 3s - loss: 0.4059 - accuracy: 0.8764
376/1688 [=====>........................] - ETA: 3s - loss: 0.3921 - accuracy: 0.8807
398/1688 [======>.......................] - ETA: 2s - loss: 0.3816 - accuracy: 0.8840
421/1688 [======>.......................] - ETA: 2s - loss: 0.3741 - accuracy: 0.8865
443/1688 [======>.......................] - ETA: 2s - loss: 0.3649 - accuracy: 0.8891
465/1688 [=======>......................] - ETA: 2s - loss: 0.3558 - accuracy: 0.8918
487/1688 [=======>......................] - ETA: 2s - loss: 0.3472 - accuracy: 0.8944
509/1688 [========>.....................] - ETA: 2s - loss: 0.3398 - accuracy: 0.8967
532/1688 [========>.....................] - ETA: 2s - loss: 0.3307 - accuracy: 0.8995
554/1688 [========>.....................] - ETA: 2s - loss: 0.3223 - accuracy: 0.9019
576/1688 [=========>....................] - ETA: 2s - loss: 0.3152 - accuracy: 0.9040
598/1688 [=========>....................] - ETA: 2s - loss: 0.3071 - accuracy: 0.9067
620/1688 [==========>...................] - ETA: 2s - loss: 0.3013 - accuracy: 0.9083
643/1688 [==========>...................] - ETA: 2s - loss: 0.2961 - accuracy: 0.9100
665/1688 [==========>...................] - ETA: 2s - loss: 0.2903 - accuracy: 0.9117
687/1688 [===========>..................] - ETA: 2s - loss: 0.2846 - accuracy: 0.9133
709/1688 [===========>..................] - ETA: 2s - loss: 0.2809 - accuracy: 0.9147
732/1688 [============>.................] - ETA: 2s - loss: 0.2761 - accuracy: 0.9162
754/1688 [============>.................] - ETA: 2s - loss: 0.2717 - accuracy: 0.9176
777/1688 [============>.................] - ETA: 2s - loss: 0.2678 - accuracy: 0.9187
800/1688 [=============>................] - ETA: 2s - loss: 0.2638 - accuracy: 0.9200
822/1688 [=============>................] - ETA: 1s - loss: 0.2603 - accuracy: 0.9211
844/1688 [==============>...............] - ETA: 1s - loss: 0.2569 - accuracy: 0.9222
866/1688 [==============>...............] - ETA: 1s - loss: 0.2531 - accuracy: 0.9233
889/1688 [==============>...............] - ETA: 1s - loss: 0.2499 - accuracy: 0.9242
911/1688 [===============>..............] - ETA: 1s - loss: 0.2466 - accuracy: 0.9252
933/1688 [===============>..............] - ETA: 1s - loss: 0.2425 - accuracy: 0.9264
956/1688 [===============>..............] - ETA: 1s - loss: 0.2390 - accuracy: 0.9276
978/1688 [================>.............] - ETA: 1s - loss: 0.2366 - accuracy: 0.9282
1000/1688 [================>.............] - ETA: 1s - loss: 0.2343 - accuracy: 0.9289
1023/1688 [=================>............] - ETA: 1s - loss: 0.2313 - accuracy: 0.9298
1045/1688 [=================>............] - ETA: 1s - loss: 0.2288 - accuracy: 0.9305
1067/1688 [=================>............] - ETA: 1s - loss: 0.2263 - accuracy: 0.9312
1089/1688 [==================>...........] - ETA: 1s - loss: 0.2248 - accuracy: 0.9318
1111/1688 [==================>...........] - ETA: 1s - loss: 0.2231 - accuracy: 0.9324
1133/1688 [===================>..........] - ETA: 1s - loss: 0.2207 - accuracy: 0.9331
1156/1688 [===================>..........] - ETA: 1s - loss: 0.2187 - accuracy: 0.9337
1178/1688 [===================>..........] - ETA: 1s - loss: 0.2166 - accuracy: 0.9344
1201/1688 [====================>.........] - ETA: 1s - loss: 0.2150 - accuracy: 0.9350
1224/1688 [====================>.........] - ETA: 1s - loss: 0.2130 - accuracy: 0.9355
1246/1688 [=====================>........] - ETA: 1s - loss: 0.2108 - accuracy: 0.9362
1269/1688 [=====================>........] - ETA: 0s - loss: 0.2092 - accuracy: 0.9367
1291/1688 [=====================>........] - ETA: 0s - loss: 0.2074 - accuracy: 0.9372
1313/1688 [======================>.......] - ETA: 0s - loss: 0.2057 - accuracy: 0.9377
1336/1688 [======================>.......] - ETA: 0s - loss: 0.2037 - accuracy: 0.9383
1358/1688 [=======================>......] - ETA: 0s - loss: 0.2020 - accuracy: 0.9387
1380/1688 [=======================>......] - ETA: 0s - loss: 0.2004 - accuracy: 0.9392
1402/1688 [=======================>......] - ETA: 0s - loss: 0.1988 - accuracy: 0.9396
1424/1688 [========================>.....] - ETA: 0s - loss: 0.1967 - accuracy: 0.9402
1446/1688 [========================>.....] - ETA: 0s - loss: 0.1958 - accuracy: 0.9405
1468/1688 [=========================>....] - ETA: 0s - loss: 0.1941 - accuracy: 0.9411
1490/1688 [=========================>....] - ETA: 0s - loss: 0.1924 - accuracy: 0.9416
1512/1688 [=========================>....] - ETA: 0s - loss: 0.1910 - accuracy: 0.9421
1534/1688 [==========================>...] - ETA: 0s - loss: 0.1895 - accuracy: 0.9425
1556/1688 [==========================>...] - ETA: 0s - loss: 0.1880 - accuracy: 0.9430
1578/1688 [===========================>..] - ETA: 0s - loss: 0.1863 - accuracy: 0.9436
1600/1688 [===========================>..] - ETA: 0s - loss: 0.1851 - accuracy: 0.9440
1621/1688 [===========================>..] - ETA: 0s - loss: 0.1842 - accuracy: 0.9442
1643/1688 [============================>.] - ETA: 0s - loss: 0.1830 - accuracy: 0.9446
1665/1688 [============================>.] - ETA: 0s - loss: 0.1821 - accuracy: 0.9449
1687/1688 [============================>.] - ETA: 0s - loss: 0.1806 - accuracy: 0.9453
1688/1688 [==============================] - 6s 3ms/step - loss: 0.1806 - accuracy: 0.9453 - val_loss: 0.0727 - val_accuracy: 0.9810
Epoch 2/10
1/1688 [..............................] - ETA: 3s - loss: 0.0056 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0590 - accuracy: 0.9810
45/1688 [..............................] - ETA: 3s - loss: 0.0637 - accuracy: 0.9778
68/1688 [>.............................] - ETA: 3s - loss: 0.0688 - accuracy: 0.9770
90/1688 [>.............................] - ETA: 3s - loss: 0.0769 - accuracy: 0.9743
112/1688 [>.............................] - ETA: 3s - loss: 0.0811 - accuracy: 0.9738
134/1688 [=>............................] - ETA: 3s - loss: 0.0809 - accuracy: 0.9736
156/1688 [=>............................] - ETA: 3s - loss: 0.0848 - accuracy: 0.9736
178/1688 [==>...........................] - ETA: 3s - loss: 0.0867 - accuracy: 0.9731
201/1688 [==>...........................] - ETA: 3s - loss: 0.0864 - accuracy: 0.9734
223/1688 [==>...........................] - ETA: 3s - loss: 0.0806 - accuracy: 0.9755
246/1688 [===>..........................] - ETA: 3s - loss: 0.0798 - accuracy: 0.9761
268/1688 [===>..........................] - ETA: 3s - loss: 0.0783 - accuracy: 0.9769
290/1688 [====>.........................] - ETA: 3s - loss: 0.0779 - accuracy: 0.9766
313/1688 [====>.........................] - ETA: 3s - loss: 0.0769 - accuracy: 0.9767
335/1688 [====>.........................] - ETA: 3s - loss: 0.0770 - accuracy: 0.9770
356/1688 [=====>........................] - ETA: 3s - loss: 0.0752 - accuracy: 0.9773
378/1688 [=====>........................] - ETA: 3s - loss: 0.0742 - accuracy: 0.9774
402/1688 [======>.......................] - ETA: 2s - loss: 0.0745 - accuracy: 0.9768
425/1688 [======>.......................] - ETA: 2s - loss: 0.0750 - accuracy: 0.9765
446/1688 [======>.......................] - ETA: 2s - loss: 0.0758 - accuracy: 0.9760
468/1688 [=======>......................] - ETA: 2s - loss: 0.0747 - accuracy: 0.9766
490/1688 [=======>......................] - ETA: 2s - loss: 0.0746 - accuracy: 0.9767
512/1688 [========>.....................] - ETA: 2s - loss: 0.0751 - accuracy: 0.9769
535/1688 [========>.....................] - ETA: 2s - loss: 0.0738 - accuracy: 0.9772
557/1688 [========>.....................] - ETA: 2s - loss: 0.0732 - accuracy: 0.9774
579/1688 [=========>....................] - ETA: 2s - loss: 0.0722 - accuracy: 0.9777
601/1688 [=========>....................] - ETA: 2s - loss: 0.0724 - accuracy: 0.9776
624/1688 [==========>...................] - ETA: 2s - loss: 0.0732 - accuracy: 0.9776
646/1688 [==========>...................] - ETA: 2s - loss: 0.0731 - accuracy: 0.9777
668/1688 [==========>...................] - ETA: 2s - loss: 0.0727 - accuracy: 0.9778
690/1688 [===========>..................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9777
712/1688 [===========>..................] - ETA: 2s - loss: 0.0720 - accuracy: 0.9777
734/1688 [============>.................] - ETA: 2s - loss: 0.0712 - accuracy: 0.9778
756/1688 [============>.................] - ETA: 2s - loss: 0.0707 - accuracy: 0.9780
778/1688 [============>.................] - ETA: 2s - loss: 0.0707 - accuracy: 0.9781
800/1688 [=============>................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9781
822/1688 [=============>................] - ETA: 1s - loss: 0.0708 - accuracy: 0.9781
844/1688 [==============>...............] - ETA: 1s - loss: 0.0708 - accuracy: 0.9779
866/1688 [==============>...............] - ETA: 1s - loss: 0.0714 - accuracy: 0.9779
888/1688 [==============>...............] - ETA: 1s - loss: 0.0726 - accuracy: 0.9775
910/1688 [===============>..............] - ETA: 1s - loss: 0.0727 - accuracy: 0.9774
932/1688 [===============>..............] - ETA: 1s - loss: 0.0736 - accuracy: 0.9774
954/1688 [===============>..............] - ETA: 1s - loss: 0.0742 - accuracy: 0.9773
976/1688 [================>.............] - ETA: 1s - loss: 0.0738 - accuracy: 0.9773
998/1688 [================>.............] - ETA: 1s - loss: 0.0732 - accuracy: 0.9775
1020/1688 [=================>............] - ETA: 1s - loss: 0.0723 - accuracy: 0.9778
1042/1688 [=================>............] - ETA: 1s - loss: 0.0724 - accuracy: 0.9778
1064/1688 [=================>............] - ETA: 1s - loss: 0.0721 - accuracy: 0.9779
1086/1688 [==================>...........] - ETA: 1s - loss: 0.0723 - accuracy: 0.9779
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0722 - accuracy: 0.9779
1130/1688 [===================>..........] - ETA: 1s - loss: 0.0718 - accuracy: 0.9780
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0719 - accuracy: 0.9780
1173/1688 [===================>..........] - ETA: 1s - loss: 0.0721 - accuracy: 0.9779
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0718 - accuracy: 0.9780
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0716 - accuracy: 0.9779
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0714 - accuracy: 0.9779
1263/1688 [=====================>........] - ETA: 0s - loss: 0.0717 - accuracy: 0.9778
1285/1688 [=====================>........] - ETA: 0s - loss: 0.0715 - accuracy: 0.9778
1306/1688 [======================>.......] - ETA: 0s - loss: 0.0714 - accuracy: 0.9779
1328/1688 [======================>.......] - ETA: 0s - loss: 0.0709 - accuracy: 0.9780
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0707 - accuracy: 0.9781
1372/1688 [=======================>......] - ETA: 0s - loss: 0.0711 - accuracy: 0.9780
1395/1688 [=======================>......] - ETA: 0s - loss: 0.0713 - accuracy: 0.9780
1416/1688 [========================>.....] - ETA: 0s - loss: 0.0708 - accuracy: 0.9782
1438/1688 [========================>.....] - ETA: 0s - loss: 0.0705 - accuracy: 0.9783
1461/1688 [========================>.....] - ETA: 0s - loss: 0.0703 - accuracy: 0.9783
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0706 - accuracy: 0.9782
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0709 - accuracy: 0.9781
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0710 - accuracy: 0.9781
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0707 - accuracy: 0.9781
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0706 - accuracy: 0.9782
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0706 - accuracy: 0.9782
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0710 - accuracy: 0.9783
1639/1688 [============================>.] - ETA: 0s - loss: 0.0705 - accuracy: 0.9784
1660/1688 [============================>.] - ETA: 0s - loss: 0.0705 - accuracy: 0.9783
1680/1688 [============================>.] - ETA: 0s - loss: 0.0703 - accuracy: 0.9783
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0703 - accuracy: 0.9783 - val_loss: 0.0785 - val_accuracy: 0.9778
Epoch 3/10
1/1688 [..............................] - ETA: 3s - loss: 0.0097 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0676 - accuracy: 0.9818
46/1688 [..............................] - ETA: 3s - loss: 0.0670 - accuracy: 0.9803
68/1688 [>.............................] - ETA: 3s - loss: 0.0618 - accuracy: 0.9812
91/1688 [>.............................] - ETA: 3s - loss: 0.0624 - accuracy: 0.9811
112/1688 [>.............................] - ETA: 3s - loss: 0.0593 - accuracy: 0.9821
133/1688 [=>............................] - ETA: 3s - loss: 0.0577 - accuracy: 0.9826
155/1688 [=>............................] - ETA: 3s - loss: 0.0541 - accuracy: 0.9833
177/1688 [==>...........................] - ETA: 3s - loss: 0.0535 - accuracy: 0.9836
199/1688 [==>...........................] - ETA: 3s - loss: 0.0555 - accuracy: 0.9826
220/1688 [==>...........................] - ETA: 3s - loss: 0.0534 - accuracy: 0.9832
242/1688 [===>..........................] - ETA: 3s - loss: 0.0512 - accuracy: 0.9839
264/1688 [===>..........................] - ETA: 3s - loss: 0.0505 - accuracy: 0.9839
286/1688 [====>.........................] - ETA: 3s - loss: 0.0501 - accuracy: 0.9842
308/1688 [====>.........................] - ETA: 3s - loss: 0.0492 - accuracy: 0.9842
330/1688 [====>.........................] - ETA: 3s - loss: 0.0497 - accuracy: 0.9838
352/1688 [=====>........................] - ETA: 3s - loss: 0.0494 - accuracy: 0.9838
373/1688 [=====>........................] - ETA: 3s - loss: 0.0477 - accuracy: 0.9844
395/1688 [======>.......................] - ETA: 3s - loss: 0.0477 - accuracy: 0.9845
416/1688 [======>.......................] - ETA: 2s - loss: 0.0470 - accuracy: 0.9845
438/1688 [======>.......................] - ETA: 2s - loss: 0.0485 - accuracy: 0.9842
460/1688 [=======>......................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9842
483/1688 [=======>......................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9844
505/1688 [=======>......................] - ETA: 2s - loss: 0.0479 - accuracy: 0.9846
528/1688 [========>.....................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9846
550/1688 [========>.....................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9844
574/1688 [=========>....................] - ETA: 2s - loss: 0.0482 - accuracy: 0.9844
596/1688 [=========>....................] - ETA: 2s - loss: 0.0479 - accuracy: 0.9846
618/1688 [=========>....................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9845
640/1688 [==========>...................] - ETA: 2s - loss: 0.0492 - accuracy: 0.9842
661/1688 [==========>...................] - ETA: 2s - loss: 0.0489 - accuracy: 0.9843
683/1688 [===========>..................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9842
705/1688 [===========>..................] - ETA: 2s - loss: 0.0499 - accuracy: 0.9841
728/1688 [===========>..................] - ETA: 2s - loss: 0.0501 - accuracy: 0.9840
750/1688 [============>.................] - ETA: 2s - loss: 0.0499 - accuracy: 0.9840
773/1688 [============>.................] - ETA: 2s - loss: 0.0499 - accuracy: 0.9837
794/1688 [=============>................] - ETA: 2s - loss: 0.0498 - accuracy: 0.9837
816/1688 [=============>................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9839
838/1688 [=============>................] - ETA: 1s - loss: 0.0486 - accuracy: 0.9838
860/1688 [==============>...............] - ETA: 1s - loss: 0.0487 - accuracy: 0.9838
882/1688 [==============>...............] - ETA: 1s - loss: 0.0490 - accuracy: 0.9837
904/1688 [===============>..............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9835
926/1688 [===============>..............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9832
949/1688 [===============>..............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9833
971/1688 [================>.............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9832
993/1688 [================>.............] - ETA: 1s - loss: 0.0511 - accuracy: 0.9831
1015/1688 [=================>............] - ETA: 1s - loss: 0.0507 - accuracy: 0.9832
1037/1688 [=================>............] - ETA: 1s - loss: 0.0507 - accuracy: 0.9832
1059/1688 [=================>............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9832
1081/1688 [==================>...........] - ETA: 1s - loss: 0.0504 - accuracy: 0.9833
1103/1688 [==================>...........] - ETA: 1s - loss: 0.0502 - accuracy: 0.9833
1126/1688 [===================>..........] - ETA: 1s - loss: 0.0504 - accuracy: 0.9833
1148/1688 [===================>..........] - ETA: 1s - loss: 0.0500 - accuracy: 0.9834
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0501 - accuracy: 0.9834
1192/1688 [====================>.........] - ETA: 1s - loss: 0.0496 - accuracy: 0.9836
1213/1688 [====================>.........] - ETA: 1s - loss: 0.0496 - accuracy: 0.9836
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0495 - accuracy: 0.9837
1258/1688 [=====================>........] - ETA: 0s - loss: 0.0496 - accuracy: 0.9837
1280/1688 [=====================>........] - ETA: 0s - loss: 0.0500 - accuracy: 0.9837
1304/1688 [======================>.......] - ETA: 0s - loss: 0.0504 - accuracy: 0.9836
1327/1688 [======================>.......] - ETA: 0s - loss: 0.0505 - accuracy: 0.9836
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0505 - accuracy: 0.9837
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0509 - accuracy: 0.9836
1392/1688 [=======================>......] - ETA: 0s - loss: 0.0512 - accuracy: 0.9834
1415/1688 [========================>.....] - ETA: 0s - loss: 0.0510 - accuracy: 0.9835
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0512 - accuracy: 0.9834
1459/1688 [========================>.....] - ETA: 0s - loss: 0.0512 - accuracy: 0.9834
1481/1688 [=========================>....] - ETA: 0s - loss: 0.0514 - accuracy: 0.9832
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0514 - accuracy: 0.9833
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0512 - accuracy: 0.9833
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0513 - accuracy: 0.9833
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0511 - accuracy: 0.9833
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0516 - accuracy: 0.9832
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0515 - accuracy: 0.9832
1637/1688 [============================>.] - ETA: 0s - loss: 0.0517 - accuracy: 0.9832
1659/1688 [============================>.] - ETA: 0s - loss: 0.0516 - accuracy: 0.9833
1681/1688 [============================>.] - ETA: 0s - loss: 0.0518 - accuracy: 0.9832
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0519 - accuracy: 0.9832 - val_loss: 0.1019 - val_accuracy: 0.9730
Epoch 4/10
1/1688 [..............................] - ETA: 3s - loss: 0.0305 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0557 - accuracy: 0.9864
45/1688 [..............................] - ETA: 3s - loss: 0.0491 - accuracy: 0.9896
68/1688 [>.............................] - ETA: 3s - loss: 0.0426 - accuracy: 0.9894
90/1688 [>.............................] - ETA: 3s - loss: 0.0444 - accuracy: 0.9896
112/1688 [>.............................] - ETA: 3s - loss: 0.0426 - accuracy: 0.9900
134/1688 [=>............................] - ETA: 3s - loss: 0.0391 - accuracy: 0.9900
156/1688 [=>............................] - ETA: 3s - loss: 0.0396 - accuracy: 0.9892
178/1688 [==>...........................] - ETA: 3s - loss: 0.0376 - accuracy: 0.9898
200/1688 [==>...........................] - ETA: 3s - loss: 0.0367 - accuracy: 0.9897
222/1688 [==>...........................] - ETA: 3s - loss: 0.0379 - accuracy: 0.9890
244/1688 [===>..........................] - ETA: 3s - loss: 0.0376 - accuracy: 0.9892
267/1688 [===>..........................] - ETA: 3s - loss: 0.0373 - accuracy: 0.9892
290/1688 [====>.........................] - ETA: 3s - loss: 0.0364 - accuracy: 0.9894
311/1688 [====>.........................] - ETA: 3s - loss: 0.0364 - accuracy: 0.9890
333/1688 [====>.........................] - ETA: 3s - loss: 0.0366 - accuracy: 0.9887
354/1688 [=====>........................] - ETA: 3s - loss: 0.0361 - accuracy: 0.9889
376/1688 [=====>........................] - ETA: 3s - loss: 0.0355 - accuracy: 0.9889
399/1688 [======>.......................] - ETA: 2s - loss: 0.0350 - accuracy: 0.9892
421/1688 [======>.......................] - ETA: 2s - loss: 0.0358 - accuracy: 0.9891
443/1688 [======>.......................] - ETA: 2s - loss: 0.0353 - accuracy: 0.9890
465/1688 [=======>......................] - ETA: 2s - loss: 0.0350 - accuracy: 0.9889
488/1688 [=======>......................] - ETA: 2s - loss: 0.0343 - accuracy: 0.9890
511/1688 [========>.....................] - ETA: 2s - loss: 0.0349 - accuracy: 0.9887
534/1688 [========>.....................] - ETA: 2s - loss: 0.0348 - accuracy: 0.9888
556/1688 [========>.....................] - ETA: 2s - loss: 0.0346 - accuracy: 0.9889
578/1688 [=========>....................] - ETA: 2s - loss: 0.0362 - accuracy: 0.9886
600/1688 [=========>....................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9885
622/1688 [==========>...................] - ETA: 2s - loss: 0.0367 - accuracy: 0.9886
644/1688 [==========>...................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9885
666/1688 [==========>...................] - ETA: 2s - loss: 0.0366 - accuracy: 0.9886
688/1688 [===========>..................] - ETA: 2s - loss: 0.0363 - accuracy: 0.9887
710/1688 [===========>..................] - ETA: 2s - loss: 0.0368 - accuracy: 0.9886
731/1688 [===========>..................] - ETA: 2s - loss: 0.0371 - accuracy: 0.9886
753/1688 [============>.................] - ETA: 2s - loss: 0.0374 - accuracy: 0.9886
775/1688 [============>.................] - ETA: 2s - loss: 0.0371 - accuracy: 0.9887
797/1688 [=============>................] - ETA: 2s - loss: 0.0371 - accuracy: 0.9886
819/1688 [=============>................] - ETA: 2s - loss: 0.0373 - accuracy: 0.9886
842/1688 [=============>................] - ETA: 1s - loss: 0.0381 - accuracy: 0.9883
864/1688 [==============>...............] - ETA: 1s - loss: 0.0384 - accuracy: 0.9880
886/1688 [==============>...............] - ETA: 1s - loss: 0.0384 - accuracy: 0.9880
909/1688 [===============>..............] - ETA: 1s - loss: 0.0390 - accuracy: 0.9877
931/1688 [===============>..............] - ETA: 1s - loss: 0.0392 - accuracy: 0.9875
953/1688 [===============>..............] - ETA: 1s - loss: 0.0389 - accuracy: 0.9877
976/1688 [================>.............] - ETA: 1s - loss: 0.0394 - accuracy: 0.9875
998/1688 [================>.............] - ETA: 1s - loss: 0.0393 - accuracy: 0.9876
1020/1688 [=================>............] - ETA: 1s - loss: 0.0398 - accuracy: 0.9875
1043/1688 [=================>............] - ETA: 1s - loss: 0.0397 - accuracy: 0.9875
1065/1688 [=================>............] - ETA: 1s - loss: 0.0392 - accuracy: 0.9876
1087/1688 [==================>...........] - ETA: 1s - loss: 0.0398 - accuracy: 0.9874
1109/1688 [==================>...........] - ETA: 1s - loss: 0.0396 - accuracy: 0.9875
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0399 - accuracy: 0.9875
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9875
1175/1688 [===================>..........] - ETA: 1s - loss: 0.0399 - accuracy: 0.9876
1197/1688 [====================>.........] - ETA: 1s - loss: 0.0403 - accuracy: 0.9873
1220/1688 [====================>.........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9875
1242/1688 [=====================>........] - ETA: 1s - loss: 0.0402 - accuracy: 0.9874
1265/1688 [=====================>........] - ETA: 0s - loss: 0.0405 - accuracy: 0.9873
1288/1688 [=====================>........] - ETA: 0s - loss: 0.0407 - accuracy: 0.9872
1310/1688 [======================>.......] - ETA: 0s - loss: 0.0406 - accuracy: 0.9872
1332/1688 [======================>.......] - ETA: 0s - loss: 0.0408 - accuracy: 0.9871
1355/1688 [=======================>......] - ETA: 0s - loss: 0.0412 - accuracy: 0.9869
1377/1688 [=======================>......] - ETA: 0s - loss: 0.0412 - accuracy: 0.9869
1399/1688 [=======================>......] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0407 - accuracy: 0.9871
1442/1688 [========================>.....] - ETA: 0s - loss: 0.0407 - accuracy: 0.9871
1464/1688 [=========================>....] - ETA: 0s - loss: 0.0408 - accuracy: 0.9871
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0406 - accuracy: 0.9872
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0409 - accuracy: 0.9871
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0414 - accuracy: 0.9869
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0415 - accuracy: 0.9868
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0414 - accuracy: 0.9868
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0413 - accuracy: 0.9868
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0414 - accuracy: 0.9868
1640/1688 [============================>.] - ETA: 0s - loss: 0.0413 - accuracy: 0.9868
1662/1688 [============================>.] - ETA: 0s - loss: 0.0410 - accuracy: 0.9869
1684/1688 [============================>.] - ETA: 0s - loss: 0.0414 - accuracy: 0.9868
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0416 - accuracy: 0.9867 - val_loss: 0.0578 - val_accuracy: 0.9842
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 0.0104 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0229 - accuracy: 0.9932
45/1688 [..............................] - ETA: 3s - loss: 0.0218 - accuracy: 0.9944
67/1688 [>.............................] - ETA: 3s - loss: 0.0247 - accuracy: 0.9911
90/1688 [>.............................] - ETA: 3s - loss: 0.0282 - accuracy: 0.9906
113/1688 [=>............................] - ETA: 3s - loss: 0.0272 - accuracy: 0.9914
135/1688 [=>............................] - ETA: 3s - loss: 0.0243 - accuracy: 0.9926
157/1688 [=>............................] - ETA: 3s - loss: 0.0258 - accuracy: 0.9924
179/1688 [==>...........................] - ETA: 3s - loss: 0.0258 - accuracy: 0.9918
201/1688 [==>...........................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9914
224/1688 [==>...........................] - ETA: 3s - loss: 0.0258 - accuracy: 0.9916
246/1688 [===>..........................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9914
269/1688 [===>..........................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9915
291/1688 [====>.........................] - ETA: 3s - loss: 0.0280 - accuracy: 0.9908
313/1688 [====>.........................] - ETA: 3s - loss: 0.0285 - accuracy: 0.9907
335/1688 [====>.........................] - ETA: 3s - loss: 0.0293 - accuracy: 0.9905
357/1688 [=====>........................] - ETA: 3s - loss: 0.0289 - accuracy: 0.9906
379/1688 [=====>........................] - ETA: 3s - loss: 0.0303 - accuracy: 0.9904
402/1688 [======>.......................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9904
424/1688 [======>.......................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9903
446/1688 [======>.......................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9903
468/1688 [=======>......................] - ETA: 2s - loss: 0.0310 - accuracy: 0.9903
490/1688 [=======>......................] - ETA: 2s - loss: 0.0316 - accuracy: 0.9902
512/1688 [========>.....................] - ETA: 2s - loss: 0.0320 - accuracy: 0.9902
535/1688 [========>.....................] - ETA: 2s - loss: 0.0322 - accuracy: 0.9902
557/1688 [========>.....................] - ETA: 2s - loss: 0.0323 - accuracy: 0.9901
579/1688 [=========>....................] - ETA: 2s - loss: 0.0318 - accuracy: 0.9903
601/1688 [=========>....................] - ETA: 2s - loss: 0.0317 - accuracy: 0.9901
623/1688 [==========>...................] - ETA: 2s - loss: 0.0313 - accuracy: 0.9902
645/1688 [==========>...................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9904
669/1688 [==========>...................] - ETA: 2s - loss: 0.0312 - accuracy: 0.9902
691/1688 [===========>..................] - ETA: 2s - loss: 0.0314 - accuracy: 0.9901
713/1688 [===========>..................] - ETA: 2s - loss: 0.0315 - accuracy: 0.9902
735/1688 [============>.................] - ETA: 2s - loss: 0.0319 - accuracy: 0.9900
758/1688 [============>.................] - ETA: 2s - loss: 0.0318 - accuracy: 0.9900
781/1688 [============>.................] - ETA: 2s - loss: 0.0322 - accuracy: 0.9900
803/1688 [=============>................] - ETA: 2s - loss: 0.0327 - accuracy: 0.9899
825/1688 [=============>................] - ETA: 1s - loss: 0.0326 - accuracy: 0.9899
847/1688 [==============>...............] - ETA: 1s - loss: 0.0325 - accuracy: 0.9899
869/1688 [==============>...............] - ETA: 1s - loss: 0.0325 - accuracy: 0.9900
892/1688 [==============>...............] - ETA: 1s - loss: 0.0331 - accuracy: 0.9897
914/1688 [===============>..............] - ETA: 1s - loss: 0.0333 - accuracy: 0.9896
936/1688 [===============>..............] - ETA: 1s - loss: 0.0333 - accuracy: 0.9897
957/1688 [================>.............] - ETA: 1s - loss: 0.0337 - accuracy: 0.9896
979/1688 [================>.............] - ETA: 1s - loss: 0.0340 - accuracy: 0.9896
1001/1688 [================>.............] - ETA: 1s - loss: 0.0337 - accuracy: 0.9896
1024/1688 [=================>............] - ETA: 1s - loss: 0.0335 - accuracy: 0.9896
1045/1688 [=================>............] - ETA: 1s - loss: 0.0337 - accuracy: 0.9895
1067/1688 [=================>............] - ETA: 1s - loss: 0.0337 - accuracy: 0.9895
1089/1688 [==================>...........] - ETA: 1s - loss: 0.0336 - accuracy: 0.9894
1111/1688 [==================>...........] - ETA: 1s - loss: 0.0334 - accuracy: 0.9895
1133/1688 [===================>..........] - ETA: 1s - loss: 0.0340 - accuracy: 0.9894
1155/1688 [===================>..........] - ETA: 1s - loss: 0.0341 - accuracy: 0.9893
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0342 - accuracy: 0.9891
1199/1688 [====================>.........] - ETA: 1s - loss: 0.0345 - accuracy: 0.9890
1221/1688 [====================>.........] - ETA: 1s - loss: 0.0343 - accuracy: 0.9890
1242/1688 [=====================>........] - ETA: 1s - loss: 0.0341 - accuracy: 0.9891
1264/1688 [=====================>........] - ETA: 0s - loss: 0.0337 - accuracy: 0.9892
1286/1688 [=====================>........] - ETA: 0s - loss: 0.0339 - accuracy: 0.9891
1308/1688 [======================>.......] - ETA: 0s - loss: 0.0337 - accuracy: 0.9892
1330/1688 [======================>.......] - ETA: 0s - loss: 0.0336 - accuracy: 0.9892
1352/1688 [=======================>......] - ETA: 0s - loss: 0.0337 - accuracy: 0.9892
1374/1688 [=======================>......] - ETA: 0s - loss: 0.0335 - accuracy: 0.9892
1396/1688 [=======================>......] - ETA: 0s - loss: 0.0337 - accuracy: 0.9892
1418/1688 [========================>.....] - ETA: 0s - loss: 0.0335 - accuracy: 0.9892
1440/1688 [========================>.....] - ETA: 0s - loss: 0.0338 - accuracy: 0.9892
1463/1688 [=========================>....] - ETA: 0s - loss: 0.0343 - accuracy: 0.9890
1485/1688 [=========================>....] - ETA: 0s - loss: 0.0342 - accuracy: 0.9890
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0345 - accuracy: 0.9889
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0342 - accuracy: 0.9890
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0341 - accuracy: 0.9890
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0345 - accuracy: 0.9888
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0344 - accuracy: 0.9888
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0345 - accuracy: 0.9888
1639/1688 [============================>.] - ETA: 0s - loss: 0.0345 - accuracy: 0.9888
1662/1688 [============================>.] - ETA: 0s - loss: 0.0347 - accuracy: 0.9887
1684/1688 [============================>.] - ETA: 0s - loss: 0.0346 - accuracy: 0.9888
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0346 - accuracy: 0.9887 - val_loss: 0.0508 - val_accuracy: 0.9855
Epoch 6/10
1/1688 [..............................] - ETA: 4s - loss: 0.0013 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0327 - accuracy: 0.9896
47/1688 [..............................] - ETA: 3s - loss: 0.0313 - accuracy: 0.9900
69/1688 [>.............................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9918
91/1688 [>.............................] - ETA: 3s - loss: 0.0263 - accuracy: 0.9918
114/1688 [=>............................] - ETA: 3s - loss: 0.0300 - accuracy: 0.9907
136/1688 [=>............................] - ETA: 3s - loss: 0.0283 - accuracy: 0.9910
158/1688 [=>............................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9913
180/1688 [==>...........................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9913
203/1688 [==>...........................] - ETA: 3s - loss: 0.0272 - accuracy: 0.9912
224/1688 [==>...........................] - ETA: 3s - loss: 0.0288 - accuracy: 0.9905
246/1688 [===>..........................] - ETA: 3s - loss: 0.0288 - accuracy: 0.9906
269/1688 [===>..........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9908
292/1688 [====>.........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9909
314/1688 [====>.........................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9912
336/1688 [====>.........................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9912
358/1688 [=====>........................] - ETA: 3s - loss: 0.0263 - accuracy: 0.9912
380/1688 [=====>........................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9914
403/1688 [======>.......................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9913
426/1688 [======>.......................] - ETA: 2s - loss: 0.0258 - accuracy: 0.9913
448/1688 [======>.......................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9915
472/1688 [=======>......................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9916
494/1688 [=======>......................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9916
516/1688 [========>.....................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9918
539/1688 [========>.....................] - ETA: 2s - loss: 0.0255 - accuracy: 0.9914
561/1688 [========>.....................] - ETA: 2s - loss: 0.0255 - accuracy: 0.9914
583/1688 [=========>....................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9913
605/1688 [=========>....................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9913
628/1688 [==========>...................] - ETA: 2s - loss: 0.0264 - accuracy: 0.9913
651/1688 [==========>...................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9913
673/1688 [==========>...................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9911
695/1688 [===========>..................] - ETA: 2s - loss: 0.0271 - accuracy: 0.9909
717/1688 [===========>..................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9910
739/1688 [============>.................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9910
761/1688 [============>.................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9908
783/1688 [============>.................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9910
805/1688 [=============>................] - ETA: 2s - loss: 0.0264 - accuracy: 0.9910
827/1688 [=============>................] - ETA: 1s - loss: 0.0267 - accuracy: 0.9909
851/1688 [==============>...............] - ETA: 1s - loss: 0.0271 - accuracy: 0.9906
874/1688 [==============>...............] - ETA: 1s - loss: 0.0272 - accuracy: 0.9906
896/1688 [==============>...............] - ETA: 1s - loss: 0.0273 - accuracy: 0.9907
918/1688 [===============>..............] - ETA: 1s - loss: 0.0274 - accuracy: 0.9906
940/1688 [===============>..............] - ETA: 1s - loss: 0.0270 - accuracy: 0.9908
962/1688 [================>.............] - ETA: 1s - loss: 0.0272 - accuracy: 0.9908
984/1688 [================>.............] - ETA: 1s - loss: 0.0275 - accuracy: 0.9909
1005/1688 [================>.............] - ETA: 1s - loss: 0.0274 - accuracy: 0.9909
1027/1688 [=================>............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9908
1049/1688 [=================>............] - ETA: 1s - loss: 0.0278 - accuracy: 0.9907
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0277 - accuracy: 0.9908
1094/1688 [==================>...........] - ETA: 1s - loss: 0.0280 - accuracy: 0.9907
1116/1688 [==================>...........] - ETA: 1s - loss: 0.0281 - accuracy: 0.9907
1139/1688 [===================>..........] - ETA: 1s - loss: 0.0280 - accuracy: 0.9906
1161/1688 [===================>..........] - ETA: 1s - loss: 0.0280 - accuracy: 0.9907
1184/1688 [====================>.........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9907
1207/1688 [====================>.........] - ETA: 1s - loss: 0.0281 - accuracy: 0.9906
1230/1688 [====================>.........] - ETA: 1s - loss: 0.0280 - accuracy: 0.9905
1252/1688 [=====================>........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9906
1275/1688 [=====================>........] - ETA: 0s - loss: 0.0282 - accuracy: 0.9906
1298/1688 [======================>.......] - ETA: 0s - loss: 0.0283 - accuracy: 0.9906
1320/1688 [======================>.......] - ETA: 0s - loss: 0.0285 - accuracy: 0.9906
1342/1688 [======================>.......] - ETA: 0s - loss: 0.0286 - accuracy: 0.9906
1364/1688 [=======================>......] - ETA: 0s - loss: 0.0284 - accuracy: 0.9907
1386/1688 [=======================>......] - ETA: 0s - loss: 0.0285 - accuracy: 0.9907
1408/1688 [========================>.....] - ETA: 0s - loss: 0.0284 - accuracy: 0.9907
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0286 - accuracy: 0.9906
1452/1688 [========================>.....] - ETA: 0s - loss: 0.0289 - accuracy: 0.9905
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0288 - accuracy: 0.9905
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0288 - accuracy: 0.9905
1518/1688 [=========================>....] - ETA: 0s - loss: 0.0286 - accuracy: 0.9906
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0286 - accuracy: 0.9905
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0286 - accuracy: 0.9905
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0285 - accuracy: 0.9905
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0285 - accuracy: 0.9904
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0283 - accuracy: 0.9905
1650/1688 [============================>.] - ETA: 0s - loss: 0.0283 - accuracy: 0.9905
1672/1688 [============================>.] - ETA: 0s - loss: 0.0284 - accuracy: 0.9905
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0284 - accuracy: 0.9904 - val_loss: 0.0574 - val_accuracy: 0.9855
Epoch 7/10
1/1688 [..............................] - ETA: 4s - loss: 0.0011 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9946
45/1688 [..............................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9951
68/1688 [>.............................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9931
90/1688 [>.............................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9934
112/1688 [>.............................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9925
134/1688 [=>............................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9921
156/1688 [=>............................] - ETA: 3s - loss: 0.0198 - accuracy: 0.9928
178/1688 [==>...........................] - ETA: 3s - loss: 0.0186 - accuracy: 0.9933
200/1688 [==>...........................] - ETA: 3s - loss: 0.0186 - accuracy: 0.9934
222/1688 [==>...........................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9935
244/1688 [===>..........................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9940
266/1688 [===>..........................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9942
288/1688 [====>.........................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9939
311/1688 [====>.........................] - ETA: 3s - loss: 0.0173 - accuracy: 0.9941
333/1688 [====>.........................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9940
355/1688 [=====>........................] - ETA: 3s - loss: 0.0186 - accuracy: 0.9936
378/1688 [=====>........................] - ETA: 3s - loss: 0.0188 - accuracy: 0.9935
401/1688 [======>.......................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9935
423/1688 [======>.......................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9936
445/1688 [======>.......................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9935
467/1688 [=======>......................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9934
490/1688 [=======>......................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9932
512/1688 [========>.....................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9933
534/1688 [========>.....................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9932
556/1688 [========>.....................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9930
578/1688 [=========>....................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9928
600/1688 [=========>....................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9926
622/1688 [==========>...................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9925
643/1688 [==========>...................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9925
665/1688 [==========>...................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9924
687/1688 [===========>..................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9923
710/1688 [===========>..................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9922
733/1688 [============>.................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9923
755/1688 [============>.................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9923
777/1688 [============>.................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9923
799/1688 [=============>................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9922
822/1688 [=============>................] - ETA: 1s - loss: 0.0226 - accuracy: 0.9922
844/1688 [==============>...............] - ETA: 1s - loss: 0.0225 - accuracy: 0.9922
866/1688 [==============>...............] - ETA: 1s - loss: 0.0228 - accuracy: 0.9922
888/1688 [==============>...............] - ETA: 1s - loss: 0.0230 - accuracy: 0.9922
910/1688 [===============>..............] - ETA: 1s - loss: 0.0234 - accuracy: 0.9920
933/1688 [===============>..............] - ETA: 1s - loss: 0.0231 - accuracy: 0.9922
955/1688 [===============>..............] - ETA: 1s - loss: 0.0230 - accuracy: 0.9922
977/1688 [================>.............] - ETA: 1s - loss: 0.0228 - accuracy: 0.9922
999/1688 [================>.............] - ETA: 1s - loss: 0.0228 - accuracy: 0.9922
1022/1688 [=================>............] - ETA: 1s - loss: 0.0229 - accuracy: 0.9923
1045/1688 [=================>............] - ETA: 1s - loss: 0.0230 - accuracy: 0.9923
1067/1688 [=================>............] - ETA: 1s - loss: 0.0229 - accuracy: 0.9923
1089/1688 [==================>...........] - ETA: 1s - loss: 0.0231 - accuracy: 0.9922
1111/1688 [==================>...........] - ETA: 1s - loss: 0.0232 - accuracy: 0.9923
1134/1688 [===================>..........] - ETA: 1s - loss: 0.0234 - accuracy: 0.9921
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0234 - accuracy: 0.9921
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0234 - accuracy: 0.9921
1201/1688 [====================>.........] - ETA: 1s - loss: 0.0233 - accuracy: 0.9922
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0240 - accuracy: 0.9921
1245/1688 [=====================>........] - ETA: 1s - loss: 0.0240 - accuracy: 0.9921
1268/1688 [=====================>........] - ETA: 0s - loss: 0.0244 - accuracy: 0.9919
1291/1688 [=====================>........] - ETA: 0s - loss: 0.0245 - accuracy: 0.9919
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0246 - accuracy: 0.9920
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0249 - accuracy: 0.9919
1358/1688 [=======================>......] - ETA: 0s - loss: 0.0250 - accuracy: 0.9919
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0251 - accuracy: 0.9919
1403/1688 [=======================>......] - ETA: 0s - loss: 0.0251 - accuracy: 0.9919
1426/1688 [========================>.....] - ETA: 0s - loss: 0.0250 - accuracy: 0.9919
1447/1688 [========================>.....] - ETA: 0s - loss: 0.0251 - accuracy: 0.9919
1469/1688 [=========================>....] - ETA: 0s - loss: 0.0258 - accuracy: 0.9918
1491/1688 [=========================>....] - ETA: 0s - loss: 0.0259 - accuracy: 0.9918
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0259 - accuracy: 0.9917
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0259 - accuracy: 0.9917
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0259 - accuracy: 0.9916
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0258 - accuracy: 0.9917
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0256 - accuracy: 0.9918
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0257 - accuracy: 0.9918
1645/1688 [============================>.] - ETA: 0s - loss: 0.0256 - accuracy: 0.9918
1668/1688 [============================>.] - ETA: 0s - loss: 0.0255 - accuracy: 0.9919
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0254 - accuracy: 0.9919 - val_loss: 0.0567 - val_accuracy: 0.9858
Epoch 8/10
1/1688 [..............................] - ETA: 4s - loss: 0.0014 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0115 - accuracy: 0.9959
44/1688 [..............................] - ETA: 3s - loss: 0.0119 - accuracy: 0.9957
67/1688 [>.............................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9949
89/1688 [>.............................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9954
111/1688 [>.............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9952
132/1688 [=>............................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9957
155/1688 [=>............................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9952
177/1688 [==>...........................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9947
199/1688 [==>...........................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9950
221/1688 [==>...........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9951
244/1688 [===>..........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9951
267/1688 [===>..........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9951
289/1688 [====>.........................] - ETA: 3s - loss: 0.0163 - accuracy: 0.9947
312/1688 [====>.........................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9948
334/1688 [====>.........................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9945
356/1688 [=====>........................] - ETA: 3s - loss: 0.0165 - accuracy: 0.9945
379/1688 [=====>........................] - ETA: 3s - loss: 0.0170 - accuracy: 0.9943
401/1688 [======>.......................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9942
423/1688 [======>.......................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9940
446/1688 [======>.......................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9941
468/1688 [=======>......................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9941
489/1688 [=======>......................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9943
511/1688 [========>.....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9944
533/1688 [========>.....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9944
555/1688 [========>.....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9944
577/1688 [=========>....................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9943
599/1688 [=========>....................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9941
622/1688 [==========>...................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9941
644/1688 [==========>...................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9939
666/1688 [==========>...................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9940
688/1688 [===========>..................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9939
710/1688 [===========>..................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9939
733/1688 [============>.................] - ETA: 2s - loss: 0.0190 - accuracy: 0.9939
756/1688 [============>.................] - ETA: 2s - loss: 0.0197 - accuracy: 0.9936
779/1688 [============>.................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9937
801/1688 [=============>................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9937
823/1688 [=============>................] - ETA: 1s - loss: 0.0191 - accuracy: 0.9937
846/1688 [==============>...............] - ETA: 1s - loss: 0.0190 - accuracy: 0.9938
868/1688 [==============>...............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9937
891/1688 [==============>...............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9936
914/1688 [===============>..............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9936
936/1688 [===============>..............] - ETA: 1s - loss: 0.0190 - accuracy: 0.9936
958/1688 [================>.............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9935
980/1688 [================>.............] - ETA: 1s - loss: 0.0195 - accuracy: 0.9934
1004/1688 [================>.............] - ETA: 1s - loss: 0.0194 - accuracy: 0.9934
1026/1688 [=================>............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9934
1049/1688 [=================>............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9934
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0194 - accuracy: 0.9933
1095/1688 [==================>...........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9932
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0197 - accuracy: 0.9931
1139/1688 [===================>..........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9932
1161/1688 [===================>..........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9932
1183/1688 [====================>.........] - ETA: 1s - loss: 0.0198 - accuracy: 0.9931
1206/1688 [====================>.........] - ETA: 1s - loss: 0.0201 - accuracy: 0.9930
1230/1688 [====================>.........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9930
1252/1688 [=====================>........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9929
1275/1688 [=====================>........] - ETA: 0s - loss: 0.0209 - accuracy: 0.9928
1298/1688 [======================>.......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9928
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9928
1344/1688 [======================>.......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9929
1366/1688 [=======================>......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9929
1388/1688 [=======================>......] - ETA: 0s - loss: 0.0210 - accuracy: 0.9930
1410/1688 [========================>.....] - ETA: 0s - loss: 0.0212 - accuracy: 0.9929
1433/1688 [========================>.....] - ETA: 0s - loss: 0.0213 - accuracy: 0.9929
1455/1688 [========================>.....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9928
1478/1688 [=========================>....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9928
1501/1688 [=========================>....] - ETA: 0s - loss: 0.0213 - accuracy: 0.9929
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0211 - accuracy: 0.9929
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0213 - accuracy: 0.9929
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0212 - accuracy: 0.9929
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0213 - accuracy: 0.9929
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0211 - accuracy: 0.9929
1633/1688 [============================>.] - ETA: 0s - loss: 0.0211 - accuracy: 0.9929
1656/1688 [============================>.] - ETA: 0s - loss: 0.0212 - accuracy: 0.9929
1679/1688 [============================>.] - ETA: 0s - loss: 0.0212 - accuracy: 0.9929
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0212 - accuracy: 0.9929 - val_loss: 0.0600 - val_accuracy: 0.9863
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 0.0012 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9932
45/1688 [..............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9951
68/1688 [>.............................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9954
90/1688 [>.............................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9948
113/1688 [=>............................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9956
135/1688 [=>............................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9956
158/1688 [=>............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9955
180/1688 [==>...........................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9953
202/1688 [==>...........................] - ETA: 3s - loss: 0.0135 - accuracy: 0.9957
224/1688 [==>...........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9961
246/1688 [===>..........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9961
268/1688 [===>..........................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9960
290/1688 [====>.........................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9960
312/1688 [====>.........................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9959
334/1688 [====>.........................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9961
356/1688 [=====>........................] - ETA: 3s - loss: 0.0135 - accuracy: 0.9960
377/1688 [=====>........................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9959
399/1688 [======>.......................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9958
421/1688 [======>.......................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9955
443/1688 [======>.......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9955
465/1688 [=======>......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9954
488/1688 [=======>......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9953
510/1688 [========>.....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9955
532/1688 [========>.....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9955
554/1688 [========>.....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9955
577/1688 [=========>....................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9956
599/1688 [=========>....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9955
621/1688 [==========>...................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9955
643/1688 [==========>...................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9955
666/1688 [==========>...................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9954
688/1688 [===========>..................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9955
710/1688 [===========>..................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9956
732/1688 [============>.................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9955
754/1688 [============>.................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9956
776/1688 [============>.................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9955
799/1688 [=============>................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9954
821/1688 [=============>................] - ETA: 1s - loss: 0.0147 - accuracy: 0.9955
842/1688 [=============>................] - ETA: 1s - loss: 0.0146 - accuracy: 0.9955
863/1688 [==============>...............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9955
885/1688 [==============>...............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9953
907/1688 [===============>..............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9954
929/1688 [===============>..............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9953
952/1688 [===============>..............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9952
974/1688 [================>.............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9951
996/1688 [================>.............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9952
1019/1688 [=================>............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9952
1042/1688 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9952
1064/1688 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9952
1086/1688 [==================>...........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9952
1109/1688 [==================>...........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9952
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9951
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9951
1175/1688 [===================>..........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9951
1197/1688 [====================>.........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9952
1219/1688 [====================>.........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9952
1241/1688 [=====================>........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9952
1263/1688 [=====================>........] - ETA: 0s - loss: 0.0157 - accuracy: 0.9950
1285/1688 [=====================>........] - ETA: 0s - loss: 0.0156 - accuracy: 0.9951
1308/1688 [======================>.......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9949
1330/1688 [======================>.......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9949
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0164 - accuracy: 0.9948
1373/1688 [=======================>......] - ETA: 0s - loss: 0.0165 - accuracy: 0.9948
1395/1688 [=======================>......] - ETA: 0s - loss: 0.0170 - accuracy: 0.9947
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0170 - accuracy: 0.9946
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0171 - accuracy: 0.9945
1461/1688 [========================>.....] - ETA: 0s - loss: 0.0172 - accuracy: 0.9945
1483/1688 [=========================>....] - ETA: 0s - loss: 0.0172 - accuracy: 0.9944
1506/1688 [=========================>....] - ETA: 0s - loss: 0.0171 - accuracy: 0.9945
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0175 - accuracy: 0.9943
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0174 - accuracy: 0.9943
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0174 - accuracy: 0.9943
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0176 - accuracy: 0.9943
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0175 - accuracy: 0.9943
1640/1688 [============================>.] - ETA: 0s - loss: 0.0177 - accuracy: 0.9942
1662/1688 [============================>.] - ETA: 0s - loss: 0.0178 - accuracy: 0.9942
1684/1688 [============================>.] - ETA: 0s - loss: 0.0178 - accuracy: 0.9942
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0178 - accuracy: 0.9941 - val_loss: 0.0619 - val_accuracy: 0.9862
Epoch 10/10
1/1688 [..............................] - ETA: 3s - loss: 0.0203 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9948
46/1688 [..............................] - ETA: 3s - loss: 0.0095 - accuracy: 0.9966
68/1688 [>.............................] - ETA: 3s - loss: 0.0090 - accuracy: 0.9968
90/1688 [>.............................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9962
111/1688 [>.............................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9949
134/1688 [=>............................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9946
156/1688 [=>............................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9954
177/1688 [==>...........................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9956
199/1688 [==>...........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9959
221/1688 [==>...........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9959
243/1688 [===>..........................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9959
265/1688 [===>..........................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9960
287/1688 [====>.........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9961
309/1688 [====>.........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9963
331/1688 [====>.........................] - ETA: 3s - loss: 0.0120 - accuracy: 0.9963
353/1688 [=====>........................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9964
375/1688 [=====>........................] - ETA: 3s - loss: 0.0115 - accuracy: 0.9964
397/1688 [======>.......................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9961
419/1688 [======>.......................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9963
441/1688 [======>.......................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9960
463/1688 [=======>......................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9960
485/1688 [=======>......................] - ETA: 2s - loss: 0.0119 - accuracy: 0.9960
507/1688 [========>.....................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9959
529/1688 [========>.....................] - ETA: 2s - loss: 0.0122 - accuracy: 0.9957
550/1688 [========>.....................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9956
572/1688 [=========>....................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9955
594/1688 [=========>....................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9954
616/1688 [=========>....................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9953
640/1688 [==========>...................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9952
662/1688 [==========>...................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9953
684/1688 [===========>..................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9953
706/1688 [===========>..................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9953
728/1688 [===========>..................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9953
751/1688 [============>.................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9953
773/1688 [============>.................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9953
795/1688 [=============>................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9953
817/1688 [=============>................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9953
839/1688 [=============>................] - ETA: 1s - loss: 0.0140 - accuracy: 0.9952
861/1688 [==============>...............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9951
883/1688 [==============>...............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9950
905/1688 [===============>..............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9950
927/1688 [===============>..............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9949
949/1688 [===============>..............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9948
971/1688 [================>.............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9947
993/1688 [================>.............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9947
1016/1688 [=================>............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9947
1038/1688 [=================>............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9946
1059/1688 [=================>............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9946
1081/1688 [==================>...........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9946
1103/1688 [==================>...........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9946
1125/1688 [==================>...........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9946
1147/1688 [===================>..........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9944
1168/1688 [===================>..........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9944
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9944
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9944
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9943
1256/1688 [=====================>........] - ETA: 1s - loss: 0.0164 - accuracy: 0.9944
1279/1688 [=====================>........] - ETA: 0s - loss: 0.0164 - accuracy: 0.9943
1300/1688 [======================>.......] - ETA: 0s - loss: 0.0167 - accuracy: 0.9942
1322/1688 [======================>.......] - ETA: 0s - loss: 0.0166 - accuracy: 0.9942
1344/1688 [======================>.......] - ETA: 0s - loss: 0.0170 - accuracy: 0.9941
1366/1688 [=======================>......] - ETA: 0s - loss: 0.0171 - accuracy: 0.9940
1388/1688 [=======================>......] - ETA: 0s - loss: 0.0170 - accuracy: 0.9941
1410/1688 [========================>.....] - ETA: 0s - loss: 0.0169 - accuracy: 0.9941
1432/1688 [========================>.....] - ETA: 0s - loss: 0.0170 - accuracy: 0.9941
1455/1688 [========================>.....] - ETA: 0s - loss: 0.0168 - accuracy: 0.9942
1477/1688 [=========================>....] - ETA: 0s - loss: 0.0166 - accuracy: 0.9942
1500/1688 [=========================>....] - ETA: 0s - loss: 0.0166 - accuracy: 0.9942
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0169 - accuracy: 0.9941
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0169 - accuracy: 0.9942
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0167 - accuracy: 0.9942
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0167 - accuracy: 0.9942
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0167 - accuracy: 0.9942
1633/1688 [============================>.] - ETA: 0s - loss: 0.0167 - accuracy: 0.9942
1655/1688 [============================>.] - ETA: 0s - loss: 0.0166 - accuracy: 0.9942
1676/1688 [============================>.] - ETA: 0s - loss: 0.0166 - accuracy: 0.9942
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0167 - accuracy: 0.9942 - val_loss: 0.0570 - val_accuracy: 0.9853
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9828000068664551
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bit integer activations and weights. Therefore, the floating point TF-Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a TF-Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point TF-Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a TF-Keras model with custom QuantizeML quantized layers substituted for the original TF-Keras layers.
All TF-Keras API functions can be applied on this new model: summary(), compile(),
fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:577: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples. Continuing execution.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 2:25
51/1024 [>.............................] - ETA: 0s
101/1024 [=>............................] - ETA: 0s
152/1024 [===>..........................] - ETA: 0s
204/1024 [====>.........................] - ETA: 0s
256/1024 [======>.......................] - ETA: 0s
307/1024 [=======>......................] - ETA: 0s
358/1024 [=========>....................] - ETA: 0s
410/1024 [===========>..................] - ETA: 0s
462/1024 [============>.................] - ETA: 0s
513/1024 [==============>...............] - ETA: 0s
564/1024 [===============>..............] - ETA: 0s
615/1024 [=================>............] - ETA: 0s
666/1024 [==================>...........] - ETA: 0s
717/1024 [====================>.........] - ETA: 0s
769/1024 [=====================>........] - ETA: 0s
820/1024 [=======================>......] - ETA: 0s
872/1024 [========================>.....] - ETA: 0s
924/1024 [==========================>...] - ETA: 0s
975/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 984us/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 28, 28, 1)] 0
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9765999913215637
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.982200026512146
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9811999797821045
2.4. Model fine-tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine-tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine-tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 1:47:37 - loss: 0.0532 - accuracy: 0.9688
9/1688 [..............................] - ETA: 10s - loss: 0.0238 - accuracy: 0.9826
18/1688 [..............................] - ETA: 10s - loss: 0.0300 - accuracy: 0.9861
27/1688 [..............................] - ETA: 10s - loss: 0.0362 - accuracy: 0.9838
36/1688 [..............................] - ETA: 10s - loss: 0.0408 - accuracy: 0.9844
45/1688 [..............................] - ETA: 10s - loss: 0.0349 - accuracy: 0.9875
54/1688 [..............................] - ETA: 10s - loss: 0.0331 - accuracy: 0.9878
63/1688 [>.............................] - ETA: 9s - loss: 0.0341 - accuracy: 0.9871
72/1688 [>.............................] - ETA: 9s - loss: 0.0316 - accuracy: 0.9878
81/1688 [>.............................] - ETA: 9s - loss: 0.0290 - accuracy: 0.9888
90/1688 [>.............................] - ETA: 9s - loss: 0.0273 - accuracy: 0.9892
98/1688 [>.............................] - ETA: 9s - loss: 0.0275 - accuracy: 0.9888
106/1688 [>.............................] - ETA: 9s - loss: 0.0257 - accuracy: 0.9897
115/1688 [=>............................] - ETA: 9s - loss: 0.0242 - accuracy: 0.9905
124/1688 [=>............................] - ETA: 9s - loss: 0.0241 - accuracy: 0.9909
133/1688 [=>............................] - ETA: 9s - loss: 0.0229 - accuracy: 0.9915
141/1688 [=>............................] - ETA: 9s - loss: 0.0221 - accuracy: 0.9916
150/1688 [=>............................] - ETA: 9s - loss: 0.0211 - accuracy: 0.9921
159/1688 [=>............................] - ETA: 9s - loss: 0.0207 - accuracy: 0.9921
167/1688 [=>............................] - ETA: 9s - loss: 0.0210 - accuracy: 0.9920
175/1688 [==>...........................] - ETA: 9s - loss: 0.0213 - accuracy: 0.9918
184/1688 [==>...........................] - ETA: 9s - loss: 0.0214 - accuracy: 0.9920
192/1688 [==>...........................] - ETA: 9s - loss: 0.0208 - accuracy: 0.9922
201/1688 [==>...........................] - ETA: 9s - loss: 0.0209 - accuracy: 0.9922
210/1688 [==>...........................] - ETA: 9s - loss: 0.0201 - accuracy: 0.9926
218/1688 [==>...........................] - ETA: 9s - loss: 0.0196 - accuracy: 0.9928
226/1688 [===>..........................] - ETA: 9s - loss: 0.0197 - accuracy: 0.9928
235/1688 [===>..........................] - ETA: 8s - loss: 0.0193 - accuracy: 0.9930
243/1688 [===>..........................] - ETA: 8s - loss: 0.0191 - accuracy: 0.9929
252/1688 [===>..........................] - ETA: 8s - loss: 0.0186 - accuracy: 0.9932
261/1688 [===>..........................] - ETA: 8s - loss: 0.0181 - accuracy: 0.9934
270/1688 [===>..........................] - ETA: 8s - loss: 0.0183 - accuracy: 0.9933
279/1688 [===>..........................] - ETA: 8s - loss: 0.0181 - accuracy: 0.9934
288/1688 [====>.........................] - ETA: 8s - loss: 0.0183 - accuracy: 0.9935
297/1688 [====>.........................] - ETA: 8s - loss: 0.0182 - accuracy: 0.9936
306/1688 [====>.........................] - ETA: 8s - loss: 0.0184 - accuracy: 0.9936
315/1688 [====>.........................] - ETA: 8s - loss: 0.0179 - accuracy: 0.9937
324/1688 [====>.........................] - ETA: 8s - loss: 0.0176 - accuracy: 0.9939
332/1688 [====>.........................] - ETA: 8s - loss: 0.0176 - accuracy: 0.9939
341/1688 [=====>........................] - ETA: 8s - loss: 0.0175 - accuracy: 0.9940
349/1688 [=====>........................] - ETA: 8s - loss: 0.0174 - accuracy: 0.9940
358/1688 [=====>........................] - ETA: 8s - loss: 0.0172 - accuracy: 0.9942
366/1688 [=====>........................] - ETA: 8s - loss: 0.0172 - accuracy: 0.9942
374/1688 [=====>........................] - ETA: 8s - loss: 0.0173 - accuracy: 0.9942
383/1688 [=====>........................] - ETA: 8s - loss: 0.0171 - accuracy: 0.9942
392/1688 [=====>........................] - ETA: 8s - loss: 0.0173 - accuracy: 0.9941
401/1688 [======>.......................] - ETA: 7s - loss: 0.0173 - accuracy: 0.9941
410/1688 [======>.......................] - ETA: 7s - loss: 0.0170 - accuracy: 0.9942
419/1688 [======>.......................] - ETA: 7s - loss: 0.0168 - accuracy: 0.9943
428/1688 [======>.......................] - ETA: 7s - loss: 0.0167 - accuracy: 0.9944
436/1688 [======>.......................] - ETA: 7s - loss: 0.0168 - accuracy: 0.9943
445/1688 [======>.......................] - ETA: 7s - loss: 0.0170 - accuracy: 0.9942
454/1688 [=======>......................] - ETA: 7s - loss: 0.0169 - accuracy: 0.9943
463/1688 [=======>......................] - ETA: 7s - loss: 0.0167 - accuracy: 0.9943
472/1688 [=======>......................] - ETA: 7s - loss: 0.0166 - accuracy: 0.9944
481/1688 [=======>......................] - ETA: 7s - loss: 0.0164 - accuracy: 0.9945
490/1688 [=======>......................] - ETA: 7s - loss: 0.0162 - accuracy: 0.9946
499/1688 [=======>......................] - ETA: 7s - loss: 0.0160 - accuracy: 0.9947
508/1688 [========>.....................] - ETA: 7s - loss: 0.0159 - accuracy: 0.9947
516/1688 [========>.....................] - ETA: 7s - loss: 0.0158 - accuracy: 0.9947
524/1688 [========>.....................] - ETA: 7s - loss: 0.0158 - accuracy: 0.9947
532/1688 [========>.....................] - ETA: 7s - loss: 0.0158 - accuracy: 0.9947
541/1688 [========>.....................] - ETA: 7s - loss: 0.0159 - accuracy: 0.9946
550/1688 [========>.....................] - ETA: 7s - loss: 0.0161 - accuracy: 0.9945
559/1688 [========>.....................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9945
568/1688 [=========>....................] - ETA: 6s - loss: 0.0162 - accuracy: 0.9945
576/1688 [=========>....................] - ETA: 6s - loss: 0.0160 - accuracy: 0.9946
585/1688 [=========>....................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9945
593/1688 [=========>....................] - ETA: 6s - loss: 0.0159 - accuracy: 0.9945
601/1688 [=========>....................] - ETA: 6s - loss: 0.0159 - accuracy: 0.9945
609/1688 [=========>....................] - ETA: 6s - loss: 0.0158 - accuracy: 0.9946
617/1688 [=========>....................] - ETA: 6s - loss: 0.0157 - accuracy: 0.9946
626/1688 [==========>...................] - ETA: 6s - loss: 0.0155 - accuracy: 0.9947
635/1688 [==========>...................] - ETA: 6s - loss: 0.0154 - accuracy: 0.9948
644/1688 [==========>...................] - ETA: 6s - loss: 0.0155 - accuracy: 0.9947
653/1688 [==========>...................] - ETA: 6s - loss: 0.0155 - accuracy: 0.9947
661/1688 [==========>...................] - ETA: 6s - loss: 0.0156 - accuracy: 0.9947
670/1688 [==========>...................] - ETA: 6s - loss: 0.0155 - accuracy: 0.9947
679/1688 [===========>..................] - ETA: 6s - loss: 0.0154 - accuracy: 0.9948
687/1688 [===========>..................] - ETA: 6s - loss: 0.0154 - accuracy: 0.9948
696/1688 [===========>..................] - ETA: 6s - loss: 0.0154 - accuracy: 0.9948
705/1688 [===========>..................] - ETA: 6s - loss: 0.0155 - accuracy: 0.9947
714/1688 [===========>..................] - ETA: 6s - loss: 0.0155 - accuracy: 0.9947
723/1688 [===========>..................] - ETA: 5s - loss: 0.0153 - accuracy: 0.9948
732/1688 [============>.................] - ETA: 5s - loss: 0.0152 - accuracy: 0.9948
741/1688 [============>.................] - ETA: 5s - loss: 0.0152 - accuracy: 0.9949
749/1688 [============>.................] - ETA: 5s - loss: 0.0150 - accuracy: 0.9949
757/1688 [============>.................] - ETA: 5s - loss: 0.0150 - accuracy: 0.9949
765/1688 [============>.................] - ETA: 5s - loss: 0.0149 - accuracy: 0.9950
774/1688 [============>.................] - ETA: 5s - loss: 0.0148 - accuracy: 0.9950
783/1688 [============>.................] - ETA: 5s - loss: 0.0147 - accuracy: 0.9951
792/1688 [=============>................] - ETA: 5s - loss: 0.0146 - accuracy: 0.9951
801/1688 [=============>................] - ETA: 5s - loss: 0.0146 - accuracy: 0.9951
809/1688 [=============>................] - ETA: 5s - loss: 0.0145 - accuracy: 0.9951
818/1688 [=============>................] - ETA: 5s - loss: 0.0143 - accuracy: 0.9952
826/1688 [=============>................] - ETA: 5s - loss: 0.0143 - accuracy: 0.9952
834/1688 [=============>................] - ETA: 5s - loss: 0.0142 - accuracy: 0.9953
843/1688 [=============>................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9953
851/1688 [==============>...............] - ETA: 5s - loss: 0.0141 - accuracy: 0.9953
859/1688 [==============>...............] - ETA: 5s - loss: 0.0140 - accuracy: 0.9953
868/1688 [==============>...............] - ETA: 5s - loss: 0.0139 - accuracy: 0.9954
876/1688 [==============>...............] - ETA: 5s - loss: 0.0139 - accuracy: 0.9954
885/1688 [==============>...............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9953
893/1688 [==============>...............] - ETA: 4s - loss: 0.0140 - accuracy: 0.9953
902/1688 [===============>..............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9953
911/1688 [===============>..............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9953
920/1688 [===============>..............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9954
928/1688 [===============>..............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9954
936/1688 [===============>..............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9954
945/1688 [===============>..............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9954
953/1688 [===============>..............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9954
961/1688 [================>.............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9954
969/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9953
977/1688 [================>.............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9953
986/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9953
994/1688 [================>.............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9954
1002/1688 [================>.............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9954
1011/1688 [================>.............] - ETA: 4s - loss: 0.0136 - accuracy: 0.9955
1019/1688 [=================>............] - ETA: 4s - loss: 0.0135 - accuracy: 0.9955
1028/1688 [=================>............] - ETA: 4s - loss: 0.0135 - accuracy: 0.9955
1037/1688 [=================>............] - ETA: 4s - loss: 0.0135 - accuracy: 0.9955
1046/1688 [=================>............] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1055/1688 [=================>............] - ETA: 3s - loss: 0.0135 - accuracy: 0.9955
1064/1688 [=================>............] - ETA: 3s - loss: 0.0134 - accuracy: 0.9956
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1090/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1099/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1108/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1117/1688 [==================>...........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1126/1688 [===================>..........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1135/1688 [===================>..........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1144/1688 [===================>..........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9955
1153/1688 [===================>..........] - ETA: 3s - loss: 0.0134 - accuracy: 0.9956
1162/1688 [===================>..........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
1171/1688 [===================>..........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
1180/1688 [===================>..........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
1189/1688 [====================>.........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
1197/1688 [====================>.........] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0133 - accuracy: 0.9956
1214/1688 [====================>.........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9956
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1238/1688 [=====================>........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0132 - accuracy: 0.9957
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0131 - accuracy: 0.9957
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0131 - accuracy: 0.9957
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0131 - accuracy: 0.9957
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0131 - accuracy: 0.9957
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0130 - accuracy: 0.9958
1299/1688 [======================>.......] - ETA: 2s - loss: 0.0130 - accuracy: 0.9958
1307/1688 [======================>.......] - ETA: 2s - loss: 0.0129 - accuracy: 0.9958
1316/1688 [======================>.......] - ETA: 2s - loss: 0.0129 - accuracy: 0.9958
1325/1688 [======================>.......] - ETA: 2s - loss: 0.0128 - accuracy: 0.9958
1333/1688 [======================>.......] - ETA: 2s - loss: 0.0128 - accuracy: 0.9959
1342/1688 [======================>.......] - ETA: 2s - loss: 0.0128 - accuracy: 0.9959
1350/1688 [======================>.......] - ETA: 2s - loss: 0.0127 - accuracy: 0.9959
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0126 - accuracy: 0.9959
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9959
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9959
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0125 - accuracy: 0.9960
1393/1688 [=======================>......] - ETA: 1s - loss: 0.0125 - accuracy: 0.9960
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9960
1411/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9960
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9960
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9960
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9960
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9960
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9960
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9961
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9961
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9961
1489/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9961
1498/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9961
1506/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9961
1515/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9961
1524/1688 [==========================>...] - ETA: 1s - loss: 0.0121 - accuracy: 0.9961
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9961
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9961
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9961
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9961
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0119 - accuracy: 0.9962
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9962
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9962
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9962
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9962
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9962
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0117 - accuracy: 0.9963
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0117 - accuracy: 0.9963
1637/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9963
1646/1688 [============================>.] - ETA: 0s - loss: 0.0118 - accuracy: 0.9962
1655/1688 [============================>.] - ETA: 0s - loss: 0.0118 - accuracy: 0.9962
1664/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9962
1673/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9963
1682/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9963
1688/1688 [==============================] - 16s 7ms/step - loss: 0.0116 - accuracy: 0.9963 - val_loss: 0.0504 - val_accuracy: 0.9893
Epoch 2/5
1/1688 [..............................] - ETA: 10s - loss: 0.0031 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0212 - accuracy: 0.9965
18/1688 [..............................] - ETA: 10s - loss: 0.0131 - accuracy: 0.9983
27/1688 [..............................] - ETA: 10s - loss: 0.0109 - accuracy: 0.9977
35/1688 [..............................] - ETA: 10s - loss: 0.0099 - accuracy: 0.9982
44/1688 [..............................] - ETA: 10s - loss: 0.0082 - accuracy: 0.9986
52/1688 [..............................] - ETA: 10s - loss: 0.0075 - accuracy: 0.9988
61/1688 [>.............................] - ETA: 10s - loss: 0.0079 - accuracy: 0.9985
70/1688 [>.............................] - ETA: 10s - loss: 0.0088 - accuracy: 0.9978
79/1688 [>.............................] - ETA: 10s - loss: 0.0079 - accuracy: 0.9980
88/1688 [>.............................] - ETA: 9s - loss: 0.0072 - accuracy: 0.9982
97/1688 [>.............................] - ETA: 9s - loss: 0.0080 - accuracy: 0.9977
106/1688 [>.............................] - ETA: 9s - loss: 0.0076 - accuracy: 0.9979
115/1688 [=>............................] - ETA: 9s - loss: 0.0074 - accuracy: 0.9978
124/1688 [=>............................] - ETA: 9s - loss: 0.0075 - accuracy: 0.9980
133/1688 [=>............................] - ETA: 9s - loss: 0.0074 - accuracy: 0.9981
142/1688 [=>............................] - ETA: 9s - loss: 0.0072 - accuracy: 0.9982
151/1688 [=>............................] - ETA: 9s - loss: 0.0070 - accuracy: 0.9983
160/1688 [=>............................] - ETA: 9s - loss: 0.0068 - accuracy: 0.9984
168/1688 [=>............................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9985
176/1688 [==>...........................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9986
185/1688 [==>...........................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9986
194/1688 [==>...........................] - ETA: 9s - loss: 0.0065 - accuracy: 0.9987
203/1688 [==>...........................] - ETA: 9s - loss: 0.0065 - accuracy: 0.9988
212/1688 [==>...........................] - ETA: 9s - loss: 0.0065 - accuracy: 0.9987
221/1688 [==>...........................] - ETA: 9s - loss: 0.0065 - accuracy: 0.9987
230/1688 [===>..........................] - ETA: 8s - loss: 0.0069 - accuracy: 0.9985
239/1688 [===>..........................] - ETA: 8s - loss: 0.0069 - accuracy: 0.9984
248/1688 [===>..........................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9981
257/1688 [===>..........................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9982
266/1688 [===>..........................] - ETA: 8s - loss: 0.0078 - accuracy: 0.9979
274/1688 [===>..........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9978
283/1688 [====>.........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9978
292/1688 [====>.........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9978
301/1688 [====>.........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9977
310/1688 [====>.........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9978
319/1688 [====>.........................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9976
327/1688 [====>.........................] - ETA: 8s - loss: 0.0078 - accuracy: 0.9977
335/1688 [====>.........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9978
344/1688 [=====>........................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9977
353/1688 [=====>........................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9977
362/1688 [=====>........................] - ETA: 8s - loss: 0.0078 - accuracy: 0.9978
371/1688 [=====>........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9978
380/1688 [=====>........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9979
389/1688 [=====>........................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9979
398/1688 [======>.......................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9980
407/1688 [======>.......................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9979
415/1688 [======>.......................] - ETA: 7s - loss: 0.0078 - accuracy: 0.9979
424/1688 [======>.......................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9978
433/1688 [======>.......................] - ETA: 7s - loss: 0.0078 - accuracy: 0.9978
442/1688 [======>.......................] - ETA: 7s - loss: 0.0078 - accuracy: 0.9978
451/1688 [=======>......................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9979
460/1688 [=======>......................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9979
469/1688 [=======>......................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9979
478/1688 [=======>......................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9980
487/1688 [=======>......................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9980
496/1688 [=======>......................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9980
505/1688 [=======>......................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9981
513/1688 [========>.....................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9981
522/1688 [========>.....................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9980
530/1688 [========>.....................] - ETA: 7s - loss: 0.0078 - accuracy: 0.9979
539/1688 [========>.....................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9980
548/1688 [========>.....................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9980
557/1688 [========>.....................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9980
566/1688 [=========>....................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9981
574/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9981
582/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9981
590/1688 [=========>....................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9981
598/1688 [=========>....................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9982
607/1688 [=========>....................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9981
616/1688 [=========>....................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9981
624/1688 [==========>...................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9981
632/1688 [==========>...................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9981
640/1688 [==========>...................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9980
649/1688 [==========>...................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9981
658/1688 [==========>...................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9981
667/1688 [==========>...................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9981
676/1688 [===========>..................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9981
685/1688 [===========>..................] - ETA: 6s - loss: 0.0077 - accuracy: 0.9981
693/1688 [===========>..................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9981
702/1688 [===========>..................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9981
711/1688 [===========>..................] - ETA: 6s - loss: 0.0078 - accuracy: 0.9981
719/1688 [===========>..................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9981
728/1688 [===========>..................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9981
737/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9980
746/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9981
755/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9981
763/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9980
772/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9980
780/1688 [============>.................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9980
789/1688 [=============>................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9980
798/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9980
806/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9981
815/1688 [=============>................] - ETA: 5s - loss: 0.0076 - accuracy: 0.9981
824/1688 [=============>................] - ETA: 5s - loss: 0.0076 - accuracy: 0.9981
833/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9980
842/1688 [=============>................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9980
851/1688 [==============>...............] - ETA: 5s - loss: 0.0077 - accuracy: 0.9981
860/1688 [==============>...............] - ETA: 5s - loss: 0.0076 - accuracy: 0.9981
869/1688 [==============>...............] - ETA: 5s - loss: 0.0076 - accuracy: 0.9981
877/1688 [==============>...............] - ETA: 5s - loss: 0.0076 - accuracy: 0.9981
886/1688 [==============>...............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9981
895/1688 [==============>...............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9981
904/1688 [===============>..............] - ETA: 4s - loss: 0.0075 - accuracy: 0.9981
913/1688 [===============>..............] - ETA: 4s - loss: 0.0075 - accuracy: 0.9981
922/1688 [===============>..............] - ETA: 4s - loss: 0.0075 - accuracy: 0.9981
930/1688 [===============>..............] - ETA: 4s - loss: 0.0075 - accuracy: 0.9981
939/1688 [===============>..............] - ETA: 4s - loss: 0.0075 - accuracy: 0.9981
948/1688 [===============>..............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9981
957/1688 [================>.............] - ETA: 4s - loss: 0.0075 - accuracy: 0.9981
966/1688 [================>.............] - ETA: 4s - loss: 0.0075 - accuracy: 0.9982
975/1688 [================>.............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9981
983/1688 [================>.............] - ETA: 4s - loss: 0.0079 - accuracy: 0.9980
991/1688 [================>.............] - ETA: 4s - loss: 0.0079 - accuracy: 0.9980
1000/1688 [================>.............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9980
1008/1688 [================>.............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9980
1017/1688 [=================>............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9981
1026/1688 [=================>............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9981
1035/1688 [=================>............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9981
1043/1688 [=================>............] - ETA: 3s - loss: 0.0078 - accuracy: 0.9981
1051/1688 [=================>............] - ETA: 3s - loss: 0.0078 - accuracy: 0.9981
1060/1688 [=================>............] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1069/1688 [=================>............] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1078/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1086/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1094/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1103/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1112/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9981
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9981
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9981
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9981
1166/1688 [===================>..........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9981
1175/1688 [===================>..........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9981
1184/1688 [====================>.........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9982
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9982
1201/1688 [====================>.........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9982
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0075 - accuracy: 0.9982
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0075 - accuracy: 0.9982
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0076 - accuracy: 0.9982
1236/1688 [====================>.........] - ETA: 2s - loss: 0.0075 - accuracy: 0.9982
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0076 - accuracy: 0.9982
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0076 - accuracy: 0.9982
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0076 - accuracy: 0.9982
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1279/1688 [=====================>........] - ETA: 2s - loss: 0.0077 - accuracy: 0.9981
1288/1688 [=====================>........] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1312/1688 [======================>.......] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1321/1688 [======================>.......] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1330/1688 [======================>.......] - ETA: 2s - loss: 0.0077 - accuracy: 0.9981
1339/1688 [======================>.......] - ETA: 2s - loss: 0.0077 - accuracy: 0.9981
1348/1688 [======================>.......] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1356/1688 [=======================>......] - ETA: 2s - loss: 0.0076 - accuracy: 0.9981
1365/1688 [=======================>......] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1374/1688 [=======================>......] - ETA: 1s - loss: 0.0077 - accuracy: 0.9980
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0077 - accuracy: 0.9981
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0077 - accuracy: 0.9980
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0077 - accuracy: 0.9980
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9980
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1427/1688 [========================>.....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1497/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1505/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1521/1688 [==========================>...] - ETA: 1s - loss: 0.0076 - accuracy: 0.9981
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0075 - accuracy: 0.9981
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0075 - accuracy: 0.9981
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0075 - accuracy: 0.9981
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0075 - accuracy: 0.9981
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0075 - accuracy: 0.9981
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1636/1688 [============================>.] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1644/1688 [============================>.] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1653/1688 [============================>.] - ETA: 0s - loss: 0.0074 - accuracy: 0.9981
1662/1688 [============================>.] - ETA: 0s - loss: 0.0073 - accuracy: 0.9981
1670/1688 [============================>.] - ETA: 0s - loss: 0.0073 - accuracy: 0.9981
1678/1688 [============================>.] - ETA: 0s - loss: 0.0073 - accuracy: 0.9982
1687/1688 [============================>.] - ETA: 0s - loss: 0.0073 - accuracy: 0.9981
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0073 - accuracy: 0.9981 - val_loss: 0.0507 - val_accuracy: 0.9895
Epoch 3/5
1/1688 [..............................] - ETA: 11s - loss: 0.0677 - accuracy: 0.9688
10/1688 [..............................] - ETA: 10s - loss: 0.0099 - accuracy: 0.9969
19/1688 [..............................] - ETA: 10s - loss: 0.0074 - accuracy: 0.9984
28/1688 [..............................] - ETA: 10s - loss: 0.0059 - accuracy: 0.9989
36/1688 [..............................] - ETA: 10s - loss: 0.0055 - accuracy: 0.9991
45/1688 [..............................] - ETA: 10s - loss: 0.0051 - accuracy: 0.9993
54/1688 [..............................] - ETA: 10s - loss: 0.0046 - accuracy: 0.9994
62/1688 [>.............................] - ETA: 10s - loss: 0.0044 - accuracy: 0.9995
70/1688 [>.............................] - ETA: 10s - loss: 0.0043 - accuracy: 0.9996
79/1688 [>.............................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9996
88/1688 [>.............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9996
97/1688 [>.............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9994
105/1688 [>.............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9991
114/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9992
122/1688 [=>............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9992
131/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9990
139/1688 [=>............................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9989
147/1688 [=>............................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9989
156/1688 [=>............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9990
164/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9990
173/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9991
182/1688 [==>...........................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9988
191/1688 [==>...........................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9989
200/1688 [==>...........................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9987
209/1688 [==>...........................] - ETA: 9s - loss: 0.0060 - accuracy: 0.9987
218/1688 [==>...........................] - ETA: 9s - loss: 0.0059 - accuracy: 0.9987
227/1688 [===>..........................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9988
236/1688 [===>..........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9988
244/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9988
252/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9989
260/1688 [===>..........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9989
268/1688 [===>..........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9990
276/1688 [===>..........................] - ETA: 8s - loss: 0.0059 - accuracy: 0.9989
285/1688 [====>.........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9988
293/1688 [====>.........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9988
302/1688 [====>.........................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9989
311/1688 [====>.........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9988
319/1688 [====>.........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9988
327/1688 [====>.........................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9989
336/1688 [====>.........................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9989
345/1688 [=====>........................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9989
354/1688 [=====>........................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9989
363/1688 [=====>........................] - ETA: 8s - loss: 0.0059 - accuracy: 0.9990
372/1688 [=====>........................] - ETA: 8s - loss: 0.0059 - accuracy: 0.9990
381/1688 [=====>........................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9990
389/1688 [=====>........................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9990
397/1688 [======>.......................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9991
405/1688 [======>.......................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9990
413/1688 [======>.......................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9989
422/1688 [======>.......................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9990
430/1688 [======>.......................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9990
438/1688 [======>.......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9990
446/1688 [======>.......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9990
455/1688 [=======>......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9990
463/1688 [=======>......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9991
472/1688 [=======>......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9991
480/1688 [=======>......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9991
488/1688 [=======>......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9991
497/1688 [=======>......................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9991
505/1688 [=======>......................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9991
514/1688 [========>.....................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9991
522/1688 [========>.....................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9991
531/1688 [========>.....................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9991
540/1688 [========>.....................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9991
549/1688 [========>.....................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9991
558/1688 [========>.....................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9991
567/1688 [=========>....................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9991
576/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9991
585/1688 [=========>....................] - ETA: 6s - loss: 0.0056 - accuracy: 0.9990
593/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9991
601/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9991
609/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9991
617/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9990
626/1688 [==========>...................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9991
634/1688 [==========>...................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9991
643/1688 [==========>...................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9991
652/1688 [==========>...................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9990
661/1688 [==========>...................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9991
670/1688 [==========>...................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9991
678/1688 [===========>..................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9990
687/1688 [===========>..................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9990
696/1688 [===========>..................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9990
705/1688 [===========>..................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9990
713/1688 [===========>..................] - ETA: 6s - loss: 0.0056 - accuracy: 0.9989
722/1688 [===========>..................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9989
730/1688 [===========>..................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
738/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
747/1688 [============>.................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9989
755/1688 [============>.................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9989
764/1688 [============>.................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9989
773/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
782/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
790/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
799/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
807/1688 [=============>................] - ETA: 5s - loss: 0.0054 - accuracy: 0.9990
816/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9990
825/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9990
834/1688 [=============>................] - ETA: 5s - loss: 0.0054 - accuracy: 0.9990
843/1688 [=============>................] - ETA: 5s - loss: 0.0054 - accuracy: 0.9990
851/1688 [==============>...............] - ETA: 5s - loss: 0.0054 - accuracy: 0.9990
860/1688 [==============>...............] - ETA: 5s - loss: 0.0054 - accuracy: 0.9990
869/1688 [==============>...............] - ETA: 5s - loss: 0.0054 - accuracy: 0.9990
878/1688 [==============>...............] - ETA: 5s - loss: 0.0054 - accuracy: 0.9990
887/1688 [==============>...............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
895/1688 [==============>...............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
904/1688 [===============>..............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
913/1688 [===============>..............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9989
922/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9989
931/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9989
940/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9989
948/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9989
957/1688 [================>.............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9989
965/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9989
974/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9989
983/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
992/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
1001/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
1010/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
1018/1688 [=================>............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
1027/1688 [=================>............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9990
1035/1688 [=================>............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9989
1043/1688 [=================>............] - ETA: 3s - loss: 0.0054 - accuracy: 0.9989
1051/1688 [=================>............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9989
1060/1688 [=================>............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1069/1688 [=================>............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1078/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1095/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1104/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1113/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9989
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1147/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1155/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1164/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1173/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1181/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1190/1688 [====================>.........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9989
1199/1688 [====================>.........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9989
1208/1688 [====================>.........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9988
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9988
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9988
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9988
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9988
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9988
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9988
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1349/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9989
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0056 - accuracy: 0.9989
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0056 - accuracy: 0.9989
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1437/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1463/1688 [=========================>....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9988
1471/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1480/1688 [=========================>....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9988
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1497/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9988
1505/1688 [=========================>....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9987
1514/1688 [=========================>....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9987
1523/1688 [==========================>...] - ETA: 1s - loss: 0.0057 - accuracy: 0.9987
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0057 - accuracy: 0.9987
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1638/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1646/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1654/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1663/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1672/1688 [============================>.] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1681/1688 [============================>.] - ETA: 0s - loss: 0.0059 - accuracy: 0.9985
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0059 - accuracy: 0.9985 - val_loss: 0.0487 - val_accuracy: 0.9900
Epoch 4/5
1/1688 [..............................] - ETA: 11s - loss: 0.0050 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0027 - accuracy: 1.0000
19/1688 [..............................] - ETA: 10s - loss: 0.0038 - accuracy: 0.9984
27/1688 [..............................] - ETA: 10s - loss: 0.0050 - accuracy: 0.9988
36/1688 [..............................] - ETA: 10s - loss: 0.0050 - accuracy: 0.9991
44/1688 [..............................] - ETA: 10s - loss: 0.0049 - accuracy: 0.9986
53/1688 [..............................] - ETA: 10s - loss: 0.0046 - accuracy: 0.9988
62/1688 [>.............................] - ETA: 10s - loss: 0.0059 - accuracy: 0.9985
71/1688 [>.............................] - ETA: 9s - loss: 0.0056 - accuracy: 0.9987
79/1688 [>.............................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9984
88/1688 [>.............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9986
97/1688 [>.............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9987
106/1688 [>.............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9988
115/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9989
124/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9990
133/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9991
141/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9989
150/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9990
159/1688 [=>............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9988
168/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9989
177/1688 [==>...........................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9989
186/1688 [==>...........................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9990
194/1688 [==>...........................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9990
202/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9991
210/1688 [==>...........................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9990
218/1688 [==>...........................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9990
227/1688 [===>..........................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9990
236/1688 [===>..........................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9991
244/1688 [===>..........................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9991
253/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9991
262/1688 [===>..........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9992
270/1688 [===>..........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9992
278/1688 [===>..........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9992
287/1688 [====>.........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9992
296/1688 [====>.........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9993
305/1688 [====>.........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9993
314/1688 [====>.........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9993
322/1688 [====>.........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9992
331/1688 [====>.........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9992
339/1688 [=====>........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9992
348/1688 [=====>........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9992
357/1688 [=====>........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9992
366/1688 [=====>........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9991
375/1688 [=====>........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9991
384/1688 [=====>........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9990
392/1688 [=====>........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9990
401/1688 [======>.......................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9991
410/1688 [======>.......................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9991
419/1688 [======>.......................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9991
428/1688 [======>.......................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9991
436/1688 [======>.......................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9991
444/1688 [======>.......................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9990
453/1688 [=======>......................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9990
462/1688 [=======>......................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9990
471/1688 [=======>......................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9990
479/1688 [=======>......................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9990
488/1688 [=======>......................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9990
497/1688 [=======>......................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9991
506/1688 [=======>......................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9991
514/1688 [========>.....................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9991
522/1688 [========>.....................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9990
531/1688 [========>.....................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9991
540/1688 [========>.....................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9990
549/1688 [========>.....................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9989
558/1688 [========>.....................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
567/1688 [=========>....................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
576/1688 [=========>....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
585/1688 [=========>....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
593/1688 [=========>....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
602/1688 [=========>....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
611/1688 [=========>....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
619/1688 [==========>...................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
628/1688 [==========>...................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9990
637/1688 [==========>...................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9990
646/1688 [==========>...................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
654/1688 [==========>...................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
662/1688 [==========>...................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
671/1688 [==========>...................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
680/1688 [===========>..................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
689/1688 [===========>..................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
697/1688 [===========>..................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9990
706/1688 [===========>..................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9990
714/1688 [===========>..................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
723/1688 [===========>..................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
731/1688 [===========>..................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
739/1688 [============>.................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9989
748/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
757/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
766/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
775/1688 [============>.................] - ETA: 5s - loss: 0.0048 - accuracy: 0.9989
784/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
793/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
802/1688 [=============>................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9989
810/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
819/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
827/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
836/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
845/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
854/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
863/1688 [==============>...............] - ETA: 5s - loss: 0.0048 - accuracy: 0.9989
872/1688 [==============>...............] - ETA: 5s - loss: 0.0048 - accuracy: 0.9990
880/1688 [==============>...............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
889/1688 [==============>...............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
898/1688 [==============>...............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9990
907/1688 [===============>..............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9990
916/1688 [===============>..............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9990
924/1688 [===============>..............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9990
932/1688 [===============>..............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
941/1688 [===============>..............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
950/1688 [===============>..............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9990
959/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
968/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
977/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
985/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
993/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
1002/1688 [================>.............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
1011/1688 [================>.............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
1020/1688 [=================>............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9990
1029/1688 [=================>............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
1038/1688 [=================>............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
1047/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1056/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1064/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1097/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1106/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1115/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1123/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1132/1688 [===================>..........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1141/1688 [===================>..........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1150/1688 [===================>..........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1159/1688 [===================>..........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1168/1688 [===================>..........] - ETA: 3s - loss: 0.0047 - accuracy: 0.9990
1176/1688 [===================>..........] - ETA: 3s - loss: 0.0047 - accuracy: 0.9990
1185/1688 [====================>.........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1202/1688 [====================>.........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1211/1688 [====================>.........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9990
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1229/1688 [====================>.........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9990
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9990
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9990
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9990
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9990
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9990
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1289/1688 [=====================>........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1298/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1307/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1316/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1325/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9989
1334/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9989
1343/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9989
1351/1688 [=======================>......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9989
1360/1688 [=======================>......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9989
1368/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1386/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1419/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0051 - accuracy: 0.9989
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1497/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1505/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1514/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1522/1688 [==========================>...] - ETA: 1s - loss: 0.0050 - accuracy: 0.9989
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9989
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1637/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1646/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1655/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1663/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1672/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1681/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9989
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0051 - accuracy: 0.9989 - val_loss: 0.0528 - val_accuracy: 0.9897
Epoch 5/5
1/1688 [..............................] - ETA: 9s - loss: 2.6282e-04 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0031 - accuracy: 1.0000
19/1688 [..............................] - ETA: 10s - loss: 0.0046 - accuracy: 1.0000
27/1688 [..............................] - ETA: 10s - loss: 0.0046 - accuracy: 1.0000
36/1688 [..............................] - ETA: 10s - loss: 0.0068 - accuracy: 0.9991
44/1688 [..............................] - ETA: 10s - loss: 0.0061 - accuracy: 0.9993
53/1688 [..............................] - ETA: 10s - loss: 0.0056 - accuracy: 0.9994
62/1688 [>.............................] - ETA: 10s - loss: 0.0055 - accuracy: 0.9995
70/1688 [>.............................] - ETA: 10s - loss: 0.0051 - accuracy: 0.9996
78/1688 [>.............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9996
87/1688 [>.............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9996
95/1688 [>.............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9993
104/1688 [>.............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9994
113/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9994
122/1688 [=>............................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9995
131/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9995
139/1688 [=>............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9996
147/1688 [=>............................] - ETA: 9s - loss: 0.0044 - accuracy: 0.9996
156/1688 [=>............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9994
165/1688 [=>............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9994
174/1688 [==>...........................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9991
183/1688 [==>...........................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9991
192/1688 [==>...........................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9990
200/1688 [==>...........................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9991
209/1688 [==>...........................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9991
218/1688 [==>...........................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9991
227/1688 [===>..........................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9992
236/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9991
244/1688 [===>..........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9991
253/1688 [===>..........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9991
262/1688 [===>..........................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9990
271/1688 [===>..........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9991
280/1688 [===>..........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9991
288/1688 [====>.........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9991
297/1688 [====>.........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9992
305/1688 [====>.........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9992
313/1688 [====>.........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9992
322/1688 [====>.........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9992
331/1688 [====>.........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9992
340/1688 [=====>........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9993
349/1688 [=====>........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9992
358/1688 [=====>........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9991
367/1688 [=====>........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9991
376/1688 [=====>........................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9992
385/1688 [=====>........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9992
394/1688 [======>.......................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9992
403/1688 [======>.......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9992
411/1688 [======>.......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9992
419/1688 [======>.......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
428/1688 [======>.......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
436/1688 [======>.......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
444/1688 [======>.......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9992
453/1688 [=======>......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9992
462/1688 [=======>......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9993
470/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
479/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
487/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
495/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
503/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9993
511/1688 [========>.....................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9993
520/1688 [========>.....................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9993
529/1688 [========>.....................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9994
538/1688 [========>.....................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9994
546/1688 [========>.....................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9994
555/1688 [========>.....................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9994
564/1688 [=========>....................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9994
573/1688 [=========>....................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9994
582/1688 [=========>....................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9994
590/1688 [=========>....................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9994
598/1688 [=========>....................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
606/1688 [=========>....................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
614/1688 [=========>....................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
622/1688 [==========>...................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9993
631/1688 [==========>...................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9994
639/1688 [==========>...................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9994
648/1688 [==========>...................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9994
657/1688 [==========>...................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
666/1688 [==========>...................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
674/1688 [==========>...................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
682/1688 [===========>..................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
691/1688 [===========>..................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
700/1688 [===========>..................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
709/1688 [===========>..................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
717/1688 [===========>..................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
726/1688 [===========>..................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9994
735/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9994
743/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9994
751/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9993
760/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9993
769/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9993
778/1688 [============>.................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9993
787/1688 [============>.................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9993
795/1688 [=============>................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9993
804/1688 [=============>................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9993
813/1688 [=============>................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9993
822/1688 [=============>................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9993
830/1688 [=============>................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9993
839/1688 [=============>................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9993
847/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9993
855/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9993
863/1688 [==============>...............] - ETA: 5s - loss: 0.0047 - accuracy: 0.9992
871/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
880/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
888/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
897/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
906/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
915/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
924/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
932/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
941/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
949/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
957/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
966/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
975/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
984/1688 [================>.............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9993
992/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
1000/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9993
1008/1688 [================>.............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9993
1017/1688 [=================>............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9994
1026/1688 [=================>............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9994
1035/1688 [=================>............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9994
1043/1688 [=================>............] - ETA: 3s - loss: 0.0045 - accuracy: 0.9994
1052/1688 [=================>............] - ETA: 3s - loss: 0.0045 - accuracy: 0.9994
1061/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1079/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1095/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1104/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1113/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9993
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9993
1165/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1174/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1183/1688 [====================>.........] - ETA: 3s - loss: 0.0045 - accuracy: 0.9993
1191/1688 [====================>.........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1200/1688 [====================>.........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9993
1209/1688 [====================>.........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9993
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9993
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1244/1688 [=====================>........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9992
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1271/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9992
1279/1688 [=====================>........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1288/1688 [=====================>........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1297/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1306/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1331/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1339/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1348/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1357/1688 [=======================>......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9992
1366/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1375/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9992
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9992
1427/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1454/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1463/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1489/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9992
1498/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9992
1507/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9992
1516/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1525/1688 [==========================>...] - ETA: 1s - loss: 0.0046 - accuracy: 0.9992
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0047 - accuracy: 0.9992
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0047 - accuracy: 0.9992
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0047 - accuracy: 0.9992
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0047 - accuracy: 0.9992
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9992
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1640/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1649/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1658/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1667/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1676/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1684/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9991
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0047 - accuracy: 0.9991 - val_loss: 0.0530 - val_accuracy: 0.9882
<tf_keras.src.callbacks.History object at 0x798a7efc6c50>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine-tuning:', score)
Test accuracy after fine-tuning: 0.9866999983787537
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with TF-Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test.astype(np.int32))
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9858999848365784
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-22.349201 -12.273756 -9.222633 -2.5368457 -13.935407 -9.310751
-30.686321 11.860397 -6.7832727 -1.3173655]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 14.951 seconds)