Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
32768/11490434 [..............................] - ETA: 20s
49152/11490434 [..............................] - ETA: 29s
81920/11490434 [..............................] - ETA: 29s
163840/11490434 [..............................] - ETA: 18s
212992/11490434 [..............................] - ETA: 16s
327680/11490434 [..............................] - ETA: 12s
491520/11490434 [>.............................] - ETA: 9s
761856/11490434 [>.............................] - ETA: 6s
1146880/11490434 [=>............................] - ETA: 4s
1753088/11490434 [===>..........................] - ETA: 3s
2744320/11490434 [======>.......................] - ETA: 1s
3727360/11490434 [========>.....................] - ETA: 1s
4734976/11490434 [===========>..................] - ETA: 1s
5832704/11490434 [==============>...............] - ETA: 0s
6864896/11490434 [================>.............] - ETA: 0s
7749632/11490434 [===================>..........] - ETA: 0s
8863744/11490434 [======================>.......] - ETA: 0s
9977856/11490434 [=========================>....] - ETA: 0s
11124736/11490434 [============================>.] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:02:18 - loss: 2.6604 - accuracy: 0.0938
23/1688 [..............................] - ETA: 3s - loss: 1.5612 - accuracy: 0.4891
46/1688 [..............................] - ETA: 3s - loss: 1.1189 - accuracy: 0.6440
68/1688 [>.............................] - ETA: 3s - loss: 0.9304 - accuracy: 0.7050
90/1688 [>.............................] - ETA: 3s - loss: 0.8001 - accuracy: 0.7479
112/1688 [>.............................] - ETA: 3s - loss: 0.7051 - accuracy: 0.7779
134/1688 [=>............................] - ETA: 3s - loss: 0.6393 - accuracy: 0.7994
156/1688 [=>............................] - ETA: 3s - loss: 0.5909 - accuracy: 0.8165
178/1688 [==>...........................] - ETA: 3s - loss: 0.5465 - accuracy: 0.8302
200/1688 [==>...........................] - ETA: 3s - loss: 0.5103 - accuracy: 0.8422
223/1688 [==>...........................] - ETA: 3s - loss: 0.4847 - accuracy: 0.8501
246/1688 [===>..........................] - ETA: 3s - loss: 0.4561 - accuracy: 0.8594
268/1688 [===>..........................] - ETA: 3s - loss: 0.4340 - accuracy: 0.8664
290/1688 [====>.........................] - ETA: 3s - loss: 0.4150 - accuracy: 0.8726
313/1688 [====>.........................] - ETA: 3s - loss: 0.3996 - accuracy: 0.8774
335/1688 [====>.........................] - ETA: 3s - loss: 0.3873 - accuracy: 0.8812
358/1688 [=====>........................] - ETA: 3s - loss: 0.3708 - accuracy: 0.8867
380/1688 [=====>........................] - ETA: 2s - loss: 0.3584 - accuracy: 0.8905
403/1688 [======>.......................] - ETA: 2s - loss: 0.3480 - accuracy: 0.8936
426/1688 [======>.......................] - ETA: 2s - loss: 0.3379 - accuracy: 0.8968
448/1688 [======>.......................] - ETA: 2s - loss: 0.3274 - accuracy: 0.9005
471/1688 [=======>......................] - ETA: 2s - loss: 0.3195 - accuracy: 0.9031
493/1688 [=======>......................] - ETA: 2s - loss: 0.3107 - accuracy: 0.9057
516/1688 [========>.....................] - ETA: 2s - loss: 0.3033 - accuracy: 0.9080
539/1688 [========>.....................] - ETA: 2s - loss: 0.2956 - accuracy: 0.9104
562/1688 [========>.....................] - ETA: 2s - loss: 0.2883 - accuracy: 0.9127
584/1688 [=========>....................] - ETA: 2s - loss: 0.2842 - accuracy: 0.9142
606/1688 [=========>....................] - ETA: 2s - loss: 0.2776 - accuracy: 0.9162
629/1688 [==========>...................] - ETA: 2s - loss: 0.2734 - accuracy: 0.9175
652/1688 [==========>...................] - ETA: 2s - loss: 0.2679 - accuracy: 0.9191
675/1688 [==========>...................] - ETA: 2s - loss: 0.2623 - accuracy: 0.9210
698/1688 [===========>..................] - ETA: 2s - loss: 0.2576 - accuracy: 0.9225
720/1688 [===========>..................] - ETA: 2s - loss: 0.2538 - accuracy: 0.9236
743/1688 [============>.................] - ETA: 2s - loss: 0.2500 - accuracy: 0.9247
766/1688 [============>.................] - ETA: 2s - loss: 0.2458 - accuracy: 0.9259
789/1688 [=============>................] - ETA: 2s - loss: 0.2421 - accuracy: 0.9266
812/1688 [=============>................] - ETA: 1s - loss: 0.2389 - accuracy: 0.9278
835/1688 [=============>................] - ETA: 1s - loss: 0.2354 - accuracy: 0.9287
858/1688 [==============>...............] - ETA: 1s - loss: 0.2337 - accuracy: 0.9295
880/1688 [==============>...............] - ETA: 1s - loss: 0.2305 - accuracy: 0.9304
903/1688 [===============>..............] - ETA: 1s - loss: 0.2279 - accuracy: 0.9313
925/1688 [===============>..............] - ETA: 1s - loss: 0.2253 - accuracy: 0.9320
947/1688 [===============>..............] - ETA: 1s - loss: 0.2225 - accuracy: 0.9328
969/1688 [================>.............] - ETA: 1s - loss: 0.2205 - accuracy: 0.9334
992/1688 [================>.............] - ETA: 1s - loss: 0.2172 - accuracy: 0.9345
1015/1688 [=================>............] - ETA: 1s - loss: 0.2144 - accuracy: 0.9353
1038/1688 [=================>............] - ETA: 1s - loss: 0.2116 - accuracy: 0.9361
1061/1688 [=================>............] - ETA: 1s - loss: 0.2093 - accuracy: 0.9367
1084/1688 [==================>...........] - ETA: 1s - loss: 0.2070 - accuracy: 0.9375
1107/1688 [==================>...........] - ETA: 1s - loss: 0.2046 - accuracy: 0.9381
1129/1688 [===================>..........] - ETA: 1s - loss: 0.2020 - accuracy: 0.9388
1152/1688 [===================>..........] - ETA: 1s - loss: 0.1998 - accuracy: 0.9395
1175/1688 [===================>..........] - ETA: 1s - loss: 0.1977 - accuracy: 0.9402
1198/1688 [====================>.........] - ETA: 1s - loss: 0.1957 - accuracy: 0.9408
1221/1688 [====================>.........] - ETA: 1s - loss: 0.1937 - accuracy: 0.9414
1243/1688 [=====================>........] - ETA: 1s - loss: 0.1920 - accuracy: 0.9420
1266/1688 [=====================>........] - ETA: 0s - loss: 0.1901 - accuracy: 0.9426
1288/1688 [=====================>........] - ETA: 0s - loss: 0.1888 - accuracy: 0.9431
1311/1688 [======================>.......] - ETA: 0s - loss: 0.1871 - accuracy: 0.9437
1334/1688 [======================>.......] - ETA: 0s - loss: 0.1859 - accuracy: 0.9439
1357/1688 [=======================>......] - ETA: 0s - loss: 0.1843 - accuracy: 0.9444
1380/1688 [=======================>......] - ETA: 0s - loss: 0.1826 - accuracy: 0.9450
1403/1688 [=======================>......] - ETA: 0s - loss: 0.1817 - accuracy: 0.9452
1426/1688 [========================>.....] - ETA: 0s - loss: 0.1805 - accuracy: 0.9456
1448/1688 [========================>.....] - ETA: 0s - loss: 0.1786 - accuracy: 0.9461
1471/1688 [=========================>....] - ETA: 0s - loss: 0.1775 - accuracy: 0.9465
1494/1688 [=========================>....] - ETA: 0s - loss: 0.1764 - accuracy: 0.9468
1517/1688 [=========================>....] - ETA: 0s - loss: 0.1749 - accuracy: 0.9473
1540/1688 [==========================>...] - ETA: 0s - loss: 0.1735 - accuracy: 0.9477
1563/1688 [==========================>...] - ETA: 0s - loss: 0.1723 - accuracy: 0.9480
1585/1688 [===========================>..] - ETA: 0s - loss: 0.1713 - accuracy: 0.9483
1608/1688 [===========================>..] - ETA: 0s - loss: 0.1701 - accuracy: 0.9488
1631/1688 [===========================>..] - ETA: 0s - loss: 0.1687 - accuracy: 0.9492
1654/1688 [============================>.] - ETA: 0s - loss: 0.1674 - accuracy: 0.9496
1677/1688 [============================>.] - ETA: 0s - loss: 0.1666 - accuracy: 0.9499
1688/1688 [==============================] - ETA: 0s - loss: 0.1659 - accuracy: 0.9501
1688/1688 [==============================] - 6s 3ms/step - loss: 0.1659 - accuracy: 0.9501 - val_loss: 0.1095 - val_accuracy: 0.9687
Epoch 2/10
1/1688 [..............................] - ETA: 3s - loss: 0.0341 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0658 - accuracy: 0.9740
46/1688 [..............................] - ETA: 3s - loss: 0.0838 - accuracy: 0.9728
68/1688 [>.............................] - ETA: 3s - loss: 0.0811 - accuracy: 0.9761
91/1688 [>.............................] - ETA: 3s - loss: 0.0788 - accuracy: 0.9773
114/1688 [=>............................] - ETA: 3s - loss: 0.0765 - accuracy: 0.9781
136/1688 [=>............................] - ETA: 3s - loss: 0.0711 - accuracy: 0.9798
158/1688 [=>............................] - ETA: 3s - loss: 0.0684 - accuracy: 0.9798
180/1688 [==>...........................] - ETA: 3s - loss: 0.0721 - accuracy: 0.9788
203/1688 [==>...........................] - ETA: 3s - loss: 0.0712 - accuracy: 0.9786
226/1688 [===>..........................] - ETA: 3s - loss: 0.0691 - accuracy: 0.9797
249/1688 [===>..........................] - ETA: 3s - loss: 0.0680 - accuracy: 0.9798
272/1688 [===>..........................] - ETA: 3s - loss: 0.0687 - accuracy: 0.9799
295/1688 [====>.........................] - ETA: 3s - loss: 0.0684 - accuracy: 0.9796
318/1688 [====>.........................] - ETA: 3s - loss: 0.0703 - accuracy: 0.9793
340/1688 [=====>........................] - ETA: 3s - loss: 0.0729 - accuracy: 0.9790
362/1688 [=====>........................] - ETA: 3s - loss: 0.0723 - accuracy: 0.9790
382/1688 [=====>........................] - ETA: 2s - loss: 0.0735 - accuracy: 0.9789
403/1688 [======>.......................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9791
424/1688 [======>.......................] - ETA: 2s - loss: 0.0712 - accuracy: 0.9793
446/1688 [======>.......................] - ETA: 2s - loss: 0.0704 - accuracy: 0.9794
467/1688 [=======>......................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9791
488/1688 [=======>......................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9793
509/1688 [========>.....................] - ETA: 2s - loss: 0.0701 - accuracy: 0.9792
530/1688 [========>.....................] - ETA: 2s - loss: 0.0688 - accuracy: 0.9794
551/1688 [========>.....................] - ETA: 2s - loss: 0.0680 - accuracy: 0.9797
572/1688 [=========>....................] - ETA: 2s - loss: 0.0683 - accuracy: 0.9795
593/1688 [=========>....................] - ETA: 2s - loss: 0.0688 - accuracy: 0.9792
614/1688 [=========>....................] - ETA: 2s - loss: 0.0696 - accuracy: 0.9788
635/1688 [==========>...................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9788
656/1688 [==========>...................] - ETA: 2s - loss: 0.0699 - accuracy: 0.9788
677/1688 [===========>..................] - ETA: 2s - loss: 0.0704 - accuracy: 0.9788
699/1688 [===========>..................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9787
720/1688 [===========>..................] - ETA: 2s - loss: 0.0701 - accuracy: 0.9789
741/1688 [============>.................] - ETA: 2s - loss: 0.0700 - accuracy: 0.9790
762/1688 [============>.................] - ETA: 2s - loss: 0.0700 - accuracy: 0.9789
783/1688 [============>.................] - ETA: 2s - loss: 0.0693 - accuracy: 0.9788
804/1688 [=============>................] - ETA: 2s - loss: 0.0691 - accuracy: 0.9789
825/1688 [=============>................] - ETA: 2s - loss: 0.0686 - accuracy: 0.9791
846/1688 [==============>...............] - ETA: 1s - loss: 0.0683 - accuracy: 0.9792
867/1688 [==============>...............] - ETA: 1s - loss: 0.0683 - accuracy: 0.9792
888/1688 [==============>...............] - ETA: 1s - loss: 0.0677 - accuracy: 0.9794
909/1688 [===============>..............] - ETA: 1s - loss: 0.0680 - accuracy: 0.9793
931/1688 [===============>..............] - ETA: 1s - loss: 0.0681 - accuracy: 0.9793
952/1688 [===============>..............] - ETA: 1s - loss: 0.0686 - accuracy: 0.9792
973/1688 [================>.............] - ETA: 1s - loss: 0.0686 - accuracy: 0.9792
994/1688 [================>.............] - ETA: 1s - loss: 0.0692 - accuracy: 0.9790
1015/1688 [=================>............] - ETA: 1s - loss: 0.0693 - accuracy: 0.9788
1036/1688 [=================>............] - ETA: 1s - loss: 0.0691 - accuracy: 0.9789
1057/1688 [=================>............] - ETA: 1s - loss: 0.0693 - accuracy: 0.9789
1079/1688 [==================>...........] - ETA: 1s - loss: 0.0690 - accuracy: 0.9790
1100/1688 [==================>...........] - ETA: 1s - loss: 0.0683 - accuracy: 0.9792
1121/1688 [==================>...........] - ETA: 1s - loss: 0.0685 - accuracy: 0.9791
1142/1688 [===================>..........] - ETA: 1s - loss: 0.0685 - accuracy: 0.9791
1164/1688 [===================>..........] - ETA: 1s - loss: 0.0692 - accuracy: 0.9789
1185/1688 [====================>.........] - ETA: 1s - loss: 0.0690 - accuracy: 0.9790
1207/1688 [====================>.........] - ETA: 1s - loss: 0.0692 - accuracy: 0.9790
1229/1688 [====================>.........] - ETA: 1s - loss: 0.0693 - accuracy: 0.9790
1250/1688 [=====================>........] - ETA: 1s - loss: 0.0690 - accuracy: 0.9792
1271/1688 [=====================>........] - ETA: 0s - loss: 0.0689 - accuracy: 0.9792
1292/1688 [=====================>........] - ETA: 0s - loss: 0.0689 - accuracy: 0.9793
1314/1688 [======================>.......] - ETA: 0s - loss: 0.0688 - accuracy: 0.9793
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0688 - accuracy: 0.9793
1356/1688 [=======================>......] - ETA: 0s - loss: 0.0687 - accuracy: 0.9793
1377/1688 [=======================>......] - ETA: 0s - loss: 0.0688 - accuracy: 0.9793
1398/1688 [=======================>......] - ETA: 0s - loss: 0.0688 - accuracy: 0.9793
1419/1688 [========================>.....] - ETA: 0s - loss: 0.0691 - accuracy: 0.9792
1440/1688 [========================>.....] - ETA: 0s - loss: 0.0687 - accuracy: 0.9794
1461/1688 [========================>.....] - ETA: 0s - loss: 0.0688 - accuracy: 0.9793
1482/1688 [=========================>....] - ETA: 0s - loss: 0.0683 - accuracy: 0.9795
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0680 - accuracy: 0.9796
1524/1688 [==========================>...] - ETA: 0s - loss: 0.0686 - accuracy: 0.9794
1546/1688 [==========================>...] - ETA: 0s - loss: 0.0685 - accuracy: 0.9793
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0686 - accuracy: 0.9792
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0686 - accuracy: 0.9792
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0684 - accuracy: 0.9792
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0685 - accuracy: 0.9791
1646/1688 [============================>.] - ETA: 0s - loss: 0.0682 - accuracy: 0.9792
1666/1688 [============================>.] - ETA: 0s - loss: 0.0681 - accuracy: 0.9793
1686/1688 [============================>.] - ETA: 0s - loss: 0.0683 - accuracy: 0.9792
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0684 - accuracy: 0.9792 - val_loss: 0.1018 - val_accuracy: 0.9682
Epoch 3/10
1/1688 [..............................] - ETA: 3s - loss: 0.0982 - accuracy: 0.9688
24/1688 [..............................] - ETA: 3s - loss: 0.0614 - accuracy: 0.9740
46/1688 [..............................] - ETA: 3s - loss: 0.0534 - accuracy: 0.9796
69/1688 [>.............................] - ETA: 3s - loss: 0.0492 - accuracy: 0.9819
91/1688 [>.............................] - ETA: 3s - loss: 0.0498 - accuracy: 0.9811
114/1688 [=>............................] - ETA: 3s - loss: 0.0496 - accuracy: 0.9808
137/1688 [=>............................] - ETA: 3s - loss: 0.0480 - accuracy: 0.9818
159/1688 [=>............................] - ETA: 3s - loss: 0.0463 - accuracy: 0.9823
180/1688 [==>...........................] - ETA: 3s - loss: 0.0447 - accuracy: 0.9832
201/1688 [==>...........................] - ETA: 3s - loss: 0.0433 - accuracy: 0.9840
222/1688 [==>...........................] - ETA: 3s - loss: 0.0430 - accuracy: 0.9845
244/1688 [===>..........................] - ETA: 3s - loss: 0.0434 - accuracy: 0.9849
265/1688 [===>..........................] - ETA: 3s - loss: 0.0432 - accuracy: 0.9850
286/1688 [====>.........................] - ETA: 3s - loss: 0.0437 - accuracy: 0.9850
307/1688 [====>.........................] - ETA: 3s - loss: 0.0440 - accuracy: 0.9847
328/1688 [====>.........................] - ETA: 3s - loss: 0.0426 - accuracy: 0.9854
349/1688 [=====>........................] - ETA: 3s - loss: 0.0429 - accuracy: 0.9855
370/1688 [=====>........................] - ETA: 3s - loss: 0.0424 - accuracy: 0.9856
392/1688 [=====>........................] - ETA: 3s - loss: 0.0418 - accuracy: 0.9857
413/1688 [======>.......................] - ETA: 3s - loss: 0.0418 - accuracy: 0.9856
435/1688 [======>.......................] - ETA: 2s - loss: 0.0419 - accuracy: 0.9860
456/1688 [=======>......................] - ETA: 2s - loss: 0.0422 - accuracy: 0.9860
477/1688 [=======>......................] - ETA: 2s - loss: 0.0423 - accuracy: 0.9858
498/1688 [=======>......................] - ETA: 2s - loss: 0.0424 - accuracy: 0.9857
520/1688 [========>.....................] - ETA: 2s - loss: 0.0431 - accuracy: 0.9855
541/1688 [========>.....................] - ETA: 2s - loss: 0.0443 - accuracy: 0.9849
563/1688 [=========>....................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9850
584/1688 [=========>....................] - ETA: 2s - loss: 0.0449 - accuracy: 0.9849
605/1688 [=========>....................] - ETA: 2s - loss: 0.0449 - accuracy: 0.9848
626/1688 [==========>...................] - ETA: 2s - loss: 0.0450 - accuracy: 0.9849
647/1688 [==========>...................] - ETA: 2s - loss: 0.0447 - accuracy: 0.9849
669/1688 [==========>...................] - ETA: 2s - loss: 0.0449 - accuracy: 0.9848
690/1688 [===========>..................] - ETA: 2s - loss: 0.0452 - accuracy: 0.9847
711/1688 [===========>..................] - ETA: 2s - loss: 0.0457 - accuracy: 0.9847
732/1688 [============>.................] - ETA: 2s - loss: 0.0458 - accuracy: 0.9847
753/1688 [============>.................] - ETA: 2s - loss: 0.0464 - accuracy: 0.9844
774/1688 [============>.................] - ETA: 2s - loss: 0.0471 - accuracy: 0.9843
795/1688 [=============>................] - ETA: 2s - loss: 0.0476 - accuracy: 0.9841
817/1688 [=============>................] - ETA: 2s - loss: 0.0480 - accuracy: 0.9842
838/1688 [=============>................] - ETA: 2s - loss: 0.0479 - accuracy: 0.9841
859/1688 [==============>...............] - ETA: 1s - loss: 0.0480 - accuracy: 0.9842
880/1688 [==============>...............] - ETA: 1s - loss: 0.0478 - accuracy: 0.9843
901/1688 [===============>..............] - ETA: 1s - loss: 0.0476 - accuracy: 0.9843
922/1688 [===============>..............] - ETA: 1s - loss: 0.0476 - accuracy: 0.9843
943/1688 [===============>..............] - ETA: 1s - loss: 0.0481 - accuracy: 0.9842
964/1688 [================>.............] - ETA: 1s - loss: 0.0486 - accuracy: 0.9841
985/1688 [================>.............] - ETA: 1s - loss: 0.0482 - accuracy: 0.9842
1006/1688 [================>.............] - ETA: 1s - loss: 0.0477 - accuracy: 0.9844
1027/1688 [=================>............] - ETA: 1s - loss: 0.0472 - accuracy: 0.9846
1048/1688 [=================>............] - ETA: 1s - loss: 0.0472 - accuracy: 0.9846
1069/1688 [=================>............] - ETA: 1s - loss: 0.0473 - accuracy: 0.9846
1090/1688 [==================>...........] - ETA: 1s - loss: 0.0472 - accuracy: 0.9846
1111/1688 [==================>...........] - ETA: 1s - loss: 0.0477 - accuracy: 0.9845
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0475 - accuracy: 0.9845
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0476 - accuracy: 0.9846
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0476 - accuracy: 0.9845
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0479 - accuracy: 0.9843
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0481 - accuracy: 0.9843
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0482 - accuracy: 0.9843
1258/1688 [=====================>........] - ETA: 1s - loss: 0.0481 - accuracy: 0.9844
1279/1688 [=====================>........] - ETA: 0s - loss: 0.0478 - accuracy: 0.9844
1300/1688 [======================>.......] - ETA: 0s - loss: 0.0477 - accuracy: 0.9844
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0479 - accuracy: 0.9845
1342/1688 [======================>.......] - ETA: 0s - loss: 0.0479 - accuracy: 0.9846
1364/1688 [=======================>......] - ETA: 0s - loss: 0.0480 - accuracy: 0.9845
1385/1688 [=======================>......] - ETA: 0s - loss: 0.0483 - accuracy: 0.9845
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0483 - accuracy: 0.9844
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0484 - accuracy: 0.9843
1449/1688 [========================>.....] - ETA: 0s - loss: 0.0487 - accuracy: 0.9842
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0489 - accuracy: 0.9842
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0487 - accuracy: 0.9842
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0488 - accuracy: 0.9842
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0490 - accuracy: 0.9842
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0492 - accuracy: 0.9841
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0494 - accuracy: 0.9840
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0492 - accuracy: 0.9841
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0493 - accuracy: 0.9841
1640/1688 [============================>.] - ETA: 0s - loss: 0.0492 - accuracy: 0.9842
1661/1688 [============================>.] - ETA: 0s - loss: 0.0491 - accuracy: 0.9842
1682/1688 [============================>.] - ETA: 0s - loss: 0.0489 - accuracy: 0.9843
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0489 - accuracy: 0.9843 - val_loss: 0.0634 - val_accuracy: 0.9832
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0425 - accuracy: 0.9688
22/1688 [..............................] - ETA: 3s - loss: 0.0466 - accuracy: 0.9787
43/1688 [..............................] - ETA: 3s - loss: 0.0456 - accuracy: 0.9826
64/1688 [>.............................] - ETA: 3s - loss: 0.0410 - accuracy: 0.9834
85/1688 [>.............................] - ETA: 3s - loss: 0.0385 - accuracy: 0.9853
106/1688 [>.............................] - ETA: 3s - loss: 0.0414 - accuracy: 0.9841
127/1688 [=>............................] - ETA: 3s - loss: 0.0370 - accuracy: 0.9860
148/1688 [=>............................] - ETA: 3s - loss: 0.0356 - accuracy: 0.9869
169/1688 [==>...........................] - ETA: 3s - loss: 0.0348 - accuracy: 0.9874
190/1688 [==>...........................] - ETA: 3s - loss: 0.0361 - accuracy: 0.9878
211/1688 [==>...........................] - ETA: 3s - loss: 0.0357 - accuracy: 0.9880
232/1688 [===>..........................] - ETA: 3s - loss: 0.0358 - accuracy: 0.9880
253/1688 [===>..........................] - ETA: 3s - loss: 0.0364 - accuracy: 0.9881
274/1688 [===>..........................] - ETA: 3s - loss: 0.0359 - accuracy: 0.9883
295/1688 [====>.........................] - ETA: 3s - loss: 0.0352 - accuracy: 0.9887
316/1688 [====>.........................] - ETA: 3s - loss: 0.0357 - accuracy: 0.9883
337/1688 [====>.........................] - ETA: 3s - loss: 0.0355 - accuracy: 0.9883
358/1688 [=====>........................] - ETA: 3s - loss: 0.0354 - accuracy: 0.9884
379/1688 [=====>........................] - ETA: 3s - loss: 0.0353 - accuracy: 0.9884
400/1688 [======>.......................] - ETA: 3s - loss: 0.0366 - accuracy: 0.9881
421/1688 [======>.......................] - ETA: 3s - loss: 0.0366 - accuracy: 0.9880
442/1688 [======>.......................] - ETA: 3s - loss: 0.0365 - accuracy: 0.9882
463/1688 [=======>......................] - ETA: 2s - loss: 0.0369 - accuracy: 0.9881
484/1688 [=======>......................] - ETA: 2s - loss: 0.0372 - accuracy: 0.9879
505/1688 [=======>......................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9878
526/1688 [========>.....................] - ETA: 2s - loss: 0.0364 - accuracy: 0.9879
547/1688 [========>.....................] - ETA: 2s - loss: 0.0359 - accuracy: 0.9882
568/1688 [=========>....................] - ETA: 2s - loss: 0.0356 - accuracy: 0.9882
589/1688 [=========>....................] - ETA: 2s - loss: 0.0357 - accuracy: 0.9882
610/1688 [=========>....................] - ETA: 2s - loss: 0.0355 - accuracy: 0.9884
631/1688 [==========>...................] - ETA: 2s - loss: 0.0352 - accuracy: 0.9885
652/1688 [==========>...................] - ETA: 2s - loss: 0.0352 - accuracy: 0.9885
673/1688 [==========>...................] - ETA: 2s - loss: 0.0353 - accuracy: 0.9884
694/1688 [===========>..................] - ETA: 2s - loss: 0.0357 - accuracy: 0.9883
715/1688 [===========>..................] - ETA: 2s - loss: 0.0352 - accuracy: 0.9885
736/1688 [============>.................] - ETA: 2s - loss: 0.0354 - accuracy: 0.9884
758/1688 [============>.................] - ETA: 2s - loss: 0.0352 - accuracy: 0.9885
779/1688 [============>.................] - ETA: 2s - loss: 0.0360 - accuracy: 0.9882
800/1688 [=============>................] - ETA: 2s - loss: 0.0368 - accuracy: 0.9880
821/1688 [=============>................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9879
843/1688 [=============>................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9878
864/1688 [==============>...............] - ETA: 2s - loss: 0.0371 - accuracy: 0.9877
886/1688 [==============>...............] - ETA: 1s - loss: 0.0369 - accuracy: 0.9876
907/1688 [===============>..............] - ETA: 1s - loss: 0.0369 - accuracy: 0.9877
928/1688 [===============>..............] - ETA: 1s - loss: 0.0371 - accuracy: 0.9876
949/1688 [===============>..............] - ETA: 1s - loss: 0.0370 - accuracy: 0.9876
971/1688 [================>.............] - ETA: 1s - loss: 0.0367 - accuracy: 0.9877
992/1688 [================>.............] - ETA: 1s - loss: 0.0370 - accuracy: 0.9877
1013/1688 [=================>............] - ETA: 1s - loss: 0.0376 - accuracy: 0.9875
1034/1688 [=================>............] - ETA: 1s - loss: 0.0377 - accuracy: 0.9875
1055/1688 [=================>............] - ETA: 1s - loss: 0.0378 - accuracy: 0.9874
1075/1688 [==================>...........] - ETA: 1s - loss: 0.0381 - accuracy: 0.9874
1095/1688 [==================>...........] - ETA: 1s - loss: 0.0383 - accuracy: 0.9874
1115/1688 [==================>...........] - ETA: 1s - loss: 0.0382 - accuracy: 0.9874
1134/1688 [===================>..........] - ETA: 1s - loss: 0.0384 - accuracy: 0.9873
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0385 - accuracy: 0.9873
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0385 - accuracy: 0.9873
1194/1688 [====================>.........] - ETA: 1s - loss: 0.0387 - accuracy: 0.9873
1214/1688 [====================>.........] - ETA: 1s - loss: 0.0385 - accuracy: 0.9873
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0387 - accuracy: 0.9873
1252/1688 [=====================>........] - ETA: 1s - loss: 0.0387 - accuracy: 0.9872
1272/1688 [=====================>........] - ETA: 1s - loss: 0.0388 - accuracy: 0.9872
1292/1688 [=====================>........] - ETA: 0s - loss: 0.0387 - accuracy: 0.9873
1312/1688 [======================>.......] - ETA: 0s - loss: 0.0388 - accuracy: 0.9872
1331/1688 [======================>.......] - ETA: 0s - loss: 0.0386 - accuracy: 0.9872
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0388 - accuracy: 0.9872
1371/1688 [=======================>......] - ETA: 0s - loss: 0.0386 - accuracy: 0.9872
1391/1688 [=======================>......] - ETA: 0s - loss: 0.0389 - accuracy: 0.9870
1410/1688 [========================>.....] - ETA: 0s - loss: 0.0391 - accuracy: 0.9870
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0394 - accuracy: 0.9869
1450/1688 [========================>.....] - ETA: 0s - loss: 0.0393 - accuracy: 0.9870
1470/1688 [=========================>....] - ETA: 0s - loss: 0.0395 - accuracy: 0.9870
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0393 - accuracy: 0.9870
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0394 - accuracy: 0.9870
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0395 - accuracy: 0.9869
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0397 - accuracy: 0.9868
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0399 - accuracy: 0.9867
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0398 - accuracy: 0.9867
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0401 - accuracy: 0.9866
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0399 - accuracy: 0.9867
1645/1688 [============================>.] - ETA: 0s - loss: 0.0398 - accuracy: 0.9868
1665/1688 [============================>.] - ETA: 0s - loss: 0.0398 - accuracy: 0.9868
1685/1688 [============================>.] - ETA: 0s - loss: 0.0398 - accuracy: 0.9868
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0399 - accuracy: 0.9868 - val_loss: 0.0629 - val_accuracy: 0.9820
Epoch 5/10
1/1688 [..............................] - ETA: 3s - loss: 0.0108 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0305 - accuracy: 0.9870
46/1688 [..............................] - ETA: 3s - loss: 0.0277 - accuracy: 0.9885
68/1688 [>.............................] - ETA: 3s - loss: 0.0257 - accuracy: 0.9894
90/1688 [>.............................] - ETA: 3s - loss: 0.0234 - accuracy: 0.9906
113/1688 [=>............................] - ETA: 3s - loss: 0.0240 - accuracy: 0.9898
136/1688 [=>............................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9897
158/1688 [=>............................] - ETA: 3s - loss: 0.0267 - accuracy: 0.9897
180/1688 [==>...........................] - ETA: 3s - loss: 0.0272 - accuracy: 0.9894
203/1688 [==>...........................] - ETA: 3s - loss: 0.0280 - accuracy: 0.9892
225/1688 [==>...........................] - ETA: 3s - loss: 0.0288 - accuracy: 0.9897
248/1688 [===>..........................] - ETA: 3s - loss: 0.0290 - accuracy: 0.9898
270/1688 [===>..........................] - ETA: 3s - loss: 0.0281 - accuracy: 0.9898
293/1688 [====>.........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9899
315/1688 [====>.........................] - ETA: 3s - loss: 0.0274 - accuracy: 0.9901
338/1688 [=====>........................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9903
361/1688 [=====>........................] - ETA: 3s - loss: 0.0265 - accuracy: 0.9906
383/1688 [=====>........................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9910
406/1688 [======>.......................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9911
428/1688 [======>.......................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9912
450/1688 [======>.......................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9912
473/1688 [=======>......................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9908
495/1688 [=======>......................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9909
518/1688 [========>.....................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9909
541/1688 [========>.....................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9909
563/1688 [=========>....................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9909
586/1688 [=========>....................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9911
609/1688 [=========>....................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9912
631/1688 [==========>...................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9911
654/1688 [==========>...................] - ETA: 2s - loss: 0.0255 - accuracy: 0.9911
677/1688 [===========>..................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9910
699/1688 [===========>..................] - ETA: 2s - loss: 0.0258 - accuracy: 0.9910
722/1688 [===========>..................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9909
745/1688 [============>.................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9908
768/1688 [============>.................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9908
791/1688 [=============>................] - ETA: 2s - loss: 0.0265 - accuracy: 0.9907
814/1688 [=============>................] - ETA: 1s - loss: 0.0264 - accuracy: 0.9908
837/1688 [=============>................] - ETA: 1s - loss: 0.0264 - accuracy: 0.9907
860/1688 [==============>...............] - ETA: 1s - loss: 0.0264 - accuracy: 0.9908
883/1688 [==============>...............] - ETA: 1s - loss: 0.0263 - accuracy: 0.9908
906/1688 [===============>..............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9911
927/1688 [===============>..............] - ETA: 1s - loss: 0.0261 - accuracy: 0.9911
948/1688 [===============>..............] - ETA: 1s - loss: 0.0263 - accuracy: 0.9909
969/1688 [================>.............] - ETA: 1s - loss: 0.0270 - accuracy: 0.9907
990/1688 [================>.............] - ETA: 1s - loss: 0.0270 - accuracy: 0.9907
1011/1688 [================>.............] - ETA: 1s - loss: 0.0270 - accuracy: 0.9908
1032/1688 [=================>............] - ETA: 1s - loss: 0.0275 - accuracy: 0.9905
1054/1688 [=================>............] - ETA: 1s - loss: 0.0283 - accuracy: 0.9902
1075/1688 [==================>...........] - ETA: 1s - loss: 0.0283 - accuracy: 0.9903
1096/1688 [==================>...........] - ETA: 1s - loss: 0.0282 - accuracy: 0.9903
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0278 - accuracy: 0.9905
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0282 - accuracy: 0.9904
1159/1688 [===================>..........] - ETA: 1s - loss: 0.0281 - accuracy: 0.9905
1180/1688 [===================>..........] - ETA: 1s - loss: 0.0284 - accuracy: 0.9905
1201/1688 [====================>.........] - ETA: 1s - loss: 0.0284 - accuracy: 0.9905
1222/1688 [====================>.........] - ETA: 1s - loss: 0.0285 - accuracy: 0.9904
1243/1688 [=====================>........] - ETA: 1s - loss: 0.0287 - accuracy: 0.9902
1265/1688 [=====================>........] - ETA: 0s - loss: 0.0289 - accuracy: 0.9901
1285/1688 [=====================>........] - ETA: 0s - loss: 0.0294 - accuracy: 0.9900
1307/1688 [======================>.......] - ETA: 0s - loss: 0.0296 - accuracy: 0.9900
1328/1688 [======================>.......] - ETA: 0s - loss: 0.0298 - accuracy: 0.9899
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0299 - accuracy: 0.9898
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0299 - accuracy: 0.9898
1391/1688 [=======================>......] - ETA: 0s - loss: 0.0300 - accuracy: 0.9898
1412/1688 [========================>.....] - ETA: 0s - loss: 0.0303 - accuracy: 0.9898
1433/1688 [========================>.....] - ETA: 0s - loss: 0.0304 - accuracy: 0.9897
1454/1688 [========================>.....] - ETA: 0s - loss: 0.0306 - accuracy: 0.9896
1475/1688 [=========================>....] - ETA: 0s - loss: 0.0306 - accuracy: 0.9897
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0307 - accuracy: 0.9896
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0306 - accuracy: 0.9896
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0306 - accuracy: 0.9896
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0306 - accuracy: 0.9896
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0306 - accuracy: 0.9895
1601/1688 [===========================>..] - ETA: 0s - loss: 0.0305 - accuracy: 0.9895
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0311 - accuracy: 0.9894
1643/1688 [============================>.] - ETA: 0s - loss: 0.0311 - accuracy: 0.9893
1664/1688 [============================>.] - ETA: 0s - loss: 0.0313 - accuracy: 0.9893
1685/1688 [============================>.] - ETA: 0s - loss: 0.0314 - accuracy: 0.9893
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0315 - accuracy: 0.9893 - val_loss: 0.0490 - val_accuracy: 0.9858
Epoch 6/10
1/1688 [..............................] - ETA: 3s - loss: 8.0654e-04 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0384 - accuracy: 0.9900
48/1688 [..............................] - ETA: 3s - loss: 0.0316 - accuracy: 0.9915
71/1688 [>.............................] - ETA: 3s - loss: 0.0265 - accuracy: 0.9930
94/1688 [>.............................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9927
117/1688 [=>............................] - ETA: 3s - loss: 0.0270 - accuracy: 0.9928
140/1688 [=>............................] - ETA: 3s - loss: 0.0256 - accuracy: 0.9924
163/1688 [=>............................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9921
185/1688 [==>...........................] - ETA: 3s - loss: 0.0250 - accuracy: 0.9924
206/1688 [==>...........................] - ETA: 3s - loss: 0.0235 - accuracy: 0.9930
228/1688 [===>..........................] - ETA: 3s - loss: 0.0221 - accuracy: 0.9937
249/1688 [===>..........................] - ETA: 3s - loss: 0.0223 - accuracy: 0.9933
271/1688 [===>..........................] - ETA: 3s - loss: 0.0220 - accuracy: 0.9934
293/1688 [====>.........................] - ETA: 3s - loss: 0.0228 - accuracy: 0.9933
315/1688 [====>.........................] - ETA: 3s - loss: 0.0226 - accuracy: 0.9935
337/1688 [====>.........................] - ETA: 3s - loss: 0.0223 - accuracy: 0.9933
360/1688 [=====>........................] - ETA: 3s - loss: 0.0220 - accuracy: 0.9934
383/1688 [=====>........................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9933
405/1688 [======>.......................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9934
426/1688 [======>.......................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9933
449/1688 [======>.......................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9932
472/1688 [=======>......................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9932
495/1688 [=======>......................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9932
518/1688 [========>.....................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9932
541/1688 [========>.....................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9931
564/1688 [=========>....................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9931
587/1688 [=========>....................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9930
610/1688 [=========>....................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9930
632/1688 [==========>...................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9928
654/1688 [==========>...................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9928
677/1688 [===========>..................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9928
699/1688 [===========>..................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9927
722/1688 [===========>..................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9927
744/1688 [============>.................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9926
766/1688 [============>.................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9925
788/1688 [=============>................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9925
811/1688 [=============>................] - ETA: 1s - loss: 0.0235 - accuracy: 0.9926
834/1688 [=============>................] - ETA: 1s - loss: 0.0240 - accuracy: 0.9927
857/1688 [==============>...............] - ETA: 1s - loss: 0.0241 - accuracy: 0.9927
879/1688 [==============>...............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9925
901/1688 [===============>..............] - ETA: 1s - loss: 0.0249 - accuracy: 0.9923
924/1688 [===============>..............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9921
947/1688 [===============>..............] - ETA: 1s - loss: 0.0260 - accuracy: 0.9919
970/1688 [================>.............] - ETA: 1s - loss: 0.0262 - accuracy: 0.9918
992/1688 [================>.............] - ETA: 1s - loss: 0.0263 - accuracy: 0.9919
1014/1688 [=================>............] - ETA: 1s - loss: 0.0265 - accuracy: 0.9918
1037/1688 [=================>............] - ETA: 1s - loss: 0.0263 - accuracy: 0.9918
1060/1688 [=================>............] - ETA: 1s - loss: 0.0262 - accuracy: 0.9918
1082/1688 [==================>...........] - ETA: 1s - loss: 0.0261 - accuracy: 0.9918
1105/1688 [==================>...........] - ETA: 1s - loss: 0.0261 - accuracy: 0.9917
1127/1688 [===================>..........] - ETA: 1s - loss: 0.0260 - accuracy: 0.9918
1148/1688 [===================>..........] - ETA: 1s - loss: 0.0260 - accuracy: 0.9917
1169/1688 [===================>..........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9916
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9916
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0264 - accuracy: 0.9915
1232/1688 [====================>.........] - ETA: 1s - loss: 0.0267 - accuracy: 0.9915
1253/1688 [=====================>........] - ETA: 0s - loss: 0.0267 - accuracy: 0.9915
1274/1688 [=====================>........] - ETA: 0s - loss: 0.0268 - accuracy: 0.9914
1295/1688 [======================>.......] - ETA: 0s - loss: 0.0267 - accuracy: 0.9914
1315/1688 [======================>.......] - ETA: 0s - loss: 0.0269 - accuracy: 0.9914
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0267 - accuracy: 0.9914
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0266 - accuracy: 0.9915
1378/1688 [=======================>......] - ETA: 0s - loss: 0.0267 - accuracy: 0.9914
1399/1688 [=======================>......] - ETA: 0s - loss: 0.0267 - accuracy: 0.9914
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0270 - accuracy: 0.9913
1441/1688 [========================>.....] - ETA: 0s - loss: 0.0268 - accuracy: 0.9913
1462/1688 [========================>.....] - ETA: 0s - loss: 0.0270 - accuracy: 0.9913
1483/1688 [=========================>....] - ETA: 0s - loss: 0.0270 - accuracy: 0.9912
1504/1688 [=========================>....] - ETA: 0s - loss: 0.0273 - accuracy: 0.9911
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0273 - accuracy: 0.9911
1546/1688 [==========================>...] - ETA: 0s - loss: 0.0273 - accuracy: 0.9911
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0273 - accuracy: 0.9911
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0273 - accuracy: 0.9912
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0272 - accuracy: 0.9911
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0272 - accuracy: 0.9911
1651/1688 [============================>.] - ETA: 0s - loss: 0.0271 - accuracy: 0.9911
1672/1688 [============================>.] - ETA: 0s - loss: 0.0272 - accuracy: 0.9911
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0273 - accuracy: 0.9911 - val_loss: 0.0594 - val_accuracy: 0.9835
Epoch 7/10
1/1688 [..............................] - ETA: 3s - loss: 0.0064 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0307 - accuracy: 0.9918
46/1688 [..............................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9932
68/1688 [>.............................] - ETA: 3s - loss: 0.0223 - accuracy: 0.9936
91/1688 [>.............................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9931
114/1688 [=>............................] - ETA: 3s - loss: 0.0200 - accuracy: 0.9940
136/1688 [=>............................] - ETA: 3s - loss: 0.0201 - accuracy: 0.9938
158/1688 [=>............................] - ETA: 3s - loss: 0.0186 - accuracy: 0.9943
180/1688 [==>...........................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9944
202/1688 [==>...........................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9941
224/1688 [==>...........................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9941
246/1688 [===>..........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9938
268/1688 [===>..........................] - ETA: 3s - loss: 0.0186 - accuracy: 0.9939
291/1688 [====>.........................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9936
314/1688 [====>.........................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9937
337/1688 [====>.........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9940
359/1688 [=====>........................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9940
382/1688 [=====>........................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9938
405/1688 [======>.......................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9933
427/1688 [======>.......................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9933
448/1688 [======>.......................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9931
470/1688 [=======>......................] - ETA: 2s - loss: 0.0190 - accuracy: 0.9931
491/1688 [=======>......................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9931
512/1688 [========>.....................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9932
533/1688 [========>.....................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9933
554/1688 [========>.....................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9932
576/1688 [=========>....................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9933
597/1688 [=========>....................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9931
618/1688 [=========>....................] - ETA: 2s - loss: 0.0193 - accuracy: 0.9931
639/1688 [==========>...................] - ETA: 2s - loss: 0.0193 - accuracy: 0.9931
660/1688 [==========>...................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9930
681/1688 [===========>..................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9931
702/1688 [===========>..................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9933
724/1688 [===========>..................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9933
745/1688 [============>.................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9933
766/1688 [============>.................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9933
787/1688 [============>.................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9931
808/1688 [=============>................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9931
829/1688 [=============>................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9929
849/1688 [==============>...............] - ETA: 1s - loss: 0.0205 - accuracy: 0.9928
870/1688 [==============>...............] - ETA: 1s - loss: 0.0206 - accuracy: 0.9927
891/1688 [==============>...............] - ETA: 1s - loss: 0.0204 - accuracy: 0.9927
912/1688 [===============>..............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9926
933/1688 [===============>..............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9926
954/1688 [===============>..............] - ETA: 1s - loss: 0.0208 - accuracy: 0.9927
975/1688 [================>.............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9927
996/1688 [================>.............] - ETA: 1s - loss: 0.0210 - accuracy: 0.9927
1017/1688 [=================>............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9927
1037/1688 [=================>............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9926
1057/1688 [=================>............] - ETA: 1s - loss: 0.0213 - accuracy: 0.9925
1076/1688 [==================>...........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9924
1094/1688 [==================>...........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9924
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9924
1130/1688 [===================>..........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9924
1148/1688 [===================>..........] - ETA: 1s - loss: 0.0215 - accuracy: 0.9924
1166/1688 [===================>..........] - ETA: 1s - loss: 0.0215 - accuracy: 0.9923
1184/1688 [====================>.........] - ETA: 1s - loss: 0.0216 - accuracy: 0.9922
1202/1688 [====================>.........] - ETA: 1s - loss: 0.0216 - accuracy: 0.9922
1220/1688 [====================>.........] - ETA: 1s - loss: 0.0218 - accuracy: 0.9921
1239/1688 [=====================>........] - ETA: 1s - loss: 0.0216 - accuracy: 0.9922
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0217 - accuracy: 0.9922
1275/1688 [=====================>........] - ETA: 1s - loss: 0.0217 - accuracy: 0.9922
1293/1688 [=====================>........] - ETA: 0s - loss: 0.0217 - accuracy: 0.9921
1311/1688 [======================>.......] - ETA: 0s - loss: 0.0216 - accuracy: 0.9922
1330/1688 [======================>.......] - ETA: 0s - loss: 0.0215 - accuracy: 0.9922
1348/1688 [======================>.......] - ETA: 0s - loss: 0.0216 - accuracy: 0.9922
1366/1688 [=======================>......] - ETA: 0s - loss: 0.0217 - accuracy: 0.9921
1384/1688 [=======================>......] - ETA: 0s - loss: 0.0217 - accuracy: 0.9921
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0218 - accuracy: 0.9921
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0219 - accuracy: 0.9920
1438/1688 [========================>.....] - ETA: 0s - loss: 0.0218 - accuracy: 0.9920
1456/1688 [========================>.....] - ETA: 0s - loss: 0.0222 - accuracy: 0.9920
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0223 - accuracy: 0.9919
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0223 - accuracy: 0.9920
1510/1688 [=========================>....] - ETA: 0s - loss: 0.0222 - accuracy: 0.9920
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0225 - accuracy: 0.9919
1546/1688 [==========================>...] - ETA: 0s - loss: 0.0225 - accuracy: 0.9919
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0226 - accuracy: 0.9919
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0227 - accuracy: 0.9919
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0226 - accuracy: 0.9919
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0225 - accuracy: 0.9920
1637/1688 [============================>.] - ETA: 0s - loss: 0.0224 - accuracy: 0.9920
1655/1688 [============================>.] - ETA: 0s - loss: 0.0223 - accuracy: 0.9921
1673/1688 [============================>.] - ETA: 0s - loss: 0.0224 - accuracy: 0.9921
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0223 - accuracy: 0.9921 - val_loss: 0.0571 - val_accuracy: 0.9852
Epoch 8/10
1/1688 [..............................] - ETA: 5s - loss: 0.0011 - accuracy: 1.0000
19/1688 [..............................] - ETA: 4s - loss: 0.0051 - accuracy: 1.0000
38/1688 [..............................] - ETA: 4s - loss: 0.0185 - accuracy: 0.9967
56/1688 [..............................] - ETA: 4s - loss: 0.0195 - accuracy: 0.9950
74/1688 [>.............................] - ETA: 4s - loss: 0.0171 - accuracy: 0.9954
93/1688 [>.............................] - ETA: 4s - loss: 0.0165 - accuracy: 0.9953
112/1688 [>.............................] - ETA: 4s - loss: 0.0154 - accuracy: 0.9955
130/1688 [=>............................] - ETA: 4s - loss: 0.0152 - accuracy: 0.9957
148/1688 [=>............................] - ETA: 4s - loss: 0.0143 - accuracy: 0.9960
166/1688 [=>............................] - ETA: 4s - loss: 0.0150 - accuracy: 0.9953
184/1688 [==>...........................] - ETA: 4s - loss: 0.0153 - accuracy: 0.9951
202/1688 [==>...........................] - ETA: 4s - loss: 0.0155 - accuracy: 0.9949
220/1688 [==>...........................] - ETA: 4s - loss: 0.0158 - accuracy: 0.9947
238/1688 [===>..........................] - ETA: 4s - loss: 0.0156 - accuracy: 0.9950
257/1688 [===>..........................] - ETA: 4s - loss: 0.0155 - accuracy: 0.9949
275/1688 [===>..........................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9950
293/1688 [====>.........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9951
312/1688 [====>.........................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9953
330/1688 [====>.........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9954
348/1688 [=====>........................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9952
366/1688 [=====>........................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9952
384/1688 [=====>........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9953
402/1688 [======>.......................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9953
420/1688 [======>.......................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9952
438/1688 [======>.......................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9952
456/1688 [=======>......................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9951
474/1688 [=======>......................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9949
492/1688 [=======>......................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9949
510/1688 [========>.....................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9949
529/1688 [========>.....................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9947
547/1688 [========>.....................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9946
565/1688 [=========>....................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9947
583/1688 [=========>....................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9948
601/1688 [=========>....................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9947
619/1688 [==========>...................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9948
637/1688 [==========>...................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9948
655/1688 [==========>...................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9948
674/1688 [==========>...................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9948
692/1688 [===========>..................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9948
710/1688 [===========>..................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9948
728/1688 [===========>..................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9948
747/1688 [============>.................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9946
765/1688 [============>.................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9946
783/1688 [============>.................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9947
801/1688 [=============>................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9947
819/1688 [=============>................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9947
837/1688 [=============>................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9946
855/1688 [==============>...............] - ETA: 2s - loss: 0.0159 - accuracy: 0.9946
873/1688 [==============>...............] - ETA: 2s - loss: 0.0158 - accuracy: 0.9946
891/1688 [==============>...............] - ETA: 2s - loss: 0.0158 - accuracy: 0.9946
909/1688 [===============>..............] - ETA: 2s - loss: 0.0159 - accuracy: 0.9945
928/1688 [===============>..............] - ETA: 2s - loss: 0.0164 - accuracy: 0.9944
946/1688 [===============>..............] - ETA: 2s - loss: 0.0164 - accuracy: 0.9944
964/1688 [================>.............] - ETA: 2s - loss: 0.0165 - accuracy: 0.9944
983/1688 [================>.............] - ETA: 1s - loss: 0.0167 - accuracy: 0.9943
1002/1688 [================>.............] - ETA: 1s - loss: 0.0166 - accuracy: 0.9944
1020/1688 [=================>............] - ETA: 1s - loss: 0.0170 - accuracy: 0.9941
1038/1688 [=================>............] - ETA: 1s - loss: 0.0173 - accuracy: 0.9940
1056/1688 [=================>............] - ETA: 1s - loss: 0.0176 - accuracy: 0.9938
1074/1688 [==================>...........] - ETA: 1s - loss: 0.0179 - accuracy: 0.9938
1092/1688 [==================>...........] - ETA: 1s - loss: 0.0179 - accuracy: 0.9937
1110/1688 [==================>...........] - ETA: 1s - loss: 0.0180 - accuracy: 0.9936
1128/1688 [===================>..........] - ETA: 1s - loss: 0.0179 - accuracy: 0.9936
1146/1688 [===================>..........] - ETA: 1s - loss: 0.0178 - accuracy: 0.9937
1164/1688 [===================>..........] - ETA: 1s - loss: 0.0178 - accuracy: 0.9937
1182/1688 [====================>.........] - ETA: 1s - loss: 0.0178 - accuracy: 0.9937
1200/1688 [====================>.........] - ETA: 1s - loss: 0.0178 - accuracy: 0.9937
1219/1688 [====================>.........] - ETA: 1s - loss: 0.0178 - accuracy: 0.9937
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0176 - accuracy: 0.9937
1255/1688 [=====================>........] - ETA: 1s - loss: 0.0177 - accuracy: 0.9937
1273/1688 [=====================>........] - ETA: 1s - loss: 0.0179 - accuracy: 0.9937
1291/1688 [=====================>........] - ETA: 1s - loss: 0.0180 - accuracy: 0.9936
1310/1688 [======================>.......] - ETA: 1s - loss: 0.0182 - accuracy: 0.9936
1328/1688 [======================>.......] - ETA: 1s - loss: 0.0181 - accuracy: 0.9937
1347/1688 [======================>.......] - ETA: 0s - loss: 0.0181 - accuracy: 0.9937
1365/1688 [=======================>......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9937
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9937
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9937
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0182 - accuracy: 0.9937
1438/1688 [========================>.....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9937
1456/1688 [========================>.....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9937
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9937
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0183 - accuracy: 0.9938
1510/1688 [=========================>....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9937
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0185 - accuracy: 0.9936
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1601/1688 [===========================>..] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0184 - accuracy: 0.9937
1637/1688 [============================>.] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1656/1688 [============================>.] - ETA: 0s - loss: 0.0186 - accuracy: 0.9937
1674/1688 [============================>.] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0185 - accuracy: 0.9937 - val_loss: 0.0651 - val_accuracy: 0.9857
Epoch 9/10
1/1688 [..............................] - ETA: 3s - loss: 0.0395 - accuracy: 0.9688
24/1688 [..............................] - ETA: 3s - loss: 0.0077 - accuracy: 0.9961
47/1688 [..............................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9960
70/1688 [>.............................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9946
92/1688 [>.............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9949
114/1688 [=>............................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9945
137/1688 [=>............................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9948
160/1688 [=>............................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9951
183/1688 [==>...........................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9956
205/1688 [==>...........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9956
228/1688 [===>..........................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9958
250/1688 [===>..........................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9955
272/1688 [===>..........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9956
295/1688 [====>.........................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9957
317/1688 [====>.........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9960
340/1688 [=====>........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9962
363/1688 [=====>........................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9964
386/1688 [=====>........................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9963
408/1688 [======>.......................] - ETA: 2s - loss: 0.0123 - accuracy: 0.9962
431/1688 [======>.......................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9962
454/1688 [=======>......................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9964
477/1688 [=======>......................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9960
499/1688 [=======>......................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9960
522/1688 [========>.....................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9959
545/1688 [========>.....................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9960
567/1688 [=========>....................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9959
590/1688 [=========>....................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9959
613/1688 [=========>....................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9960
635/1688 [==========>...................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9959
657/1688 [==========>...................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9959
680/1688 [===========>..................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9960
703/1688 [===========>..................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9960
726/1688 [===========>..................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9959
749/1688 [============>.................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9958
772/1688 [============>.................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9956
795/1688 [=============>................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9956
817/1688 [=============>................] - ETA: 1s - loss: 0.0133 - accuracy: 0.9956
840/1688 [=============>................] - ETA: 1s - loss: 0.0131 - accuracy: 0.9957
863/1688 [==============>...............] - ETA: 1s - loss: 0.0129 - accuracy: 0.9958
885/1688 [==============>...............] - ETA: 1s - loss: 0.0129 - accuracy: 0.9958
907/1688 [===============>..............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9957
930/1688 [===============>..............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9957
953/1688 [===============>..............] - ETA: 1s - loss: 0.0131 - accuracy: 0.9957
975/1688 [================>.............] - ETA: 1s - loss: 0.0135 - accuracy: 0.9954
997/1688 [================>.............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9953
1019/1688 [=================>............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9952
1042/1688 [=================>............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9951
1065/1688 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9950
1087/1688 [==================>...........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9949
1109/1688 [==================>...........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9948
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9948
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0153 - accuracy: 0.9947
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9947
1199/1688 [====================>.........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9947
1221/1688 [====================>.........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9947
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9945
1267/1688 [=====================>........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9945
1290/1688 [=====================>........] - ETA: 0s - loss: 0.0162 - accuracy: 0.9945
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0162 - accuracy: 0.9944
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0165 - accuracy: 0.9943
1358/1688 [=======================>......] - ETA: 0s - loss: 0.0167 - accuracy: 0.9943
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0167 - accuracy: 0.9943
1403/1688 [=======================>......] - ETA: 0s - loss: 0.0172 - accuracy: 0.9942
1426/1688 [========================>.....] - ETA: 0s - loss: 0.0175 - accuracy: 0.9941
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0174 - accuracy: 0.9941
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0173 - accuracy: 0.9941
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0173 - accuracy: 0.9941
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0174 - accuracy: 0.9941
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0173 - accuracy: 0.9941
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0172 - accuracy: 0.9941
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0171 - accuracy: 0.9942
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0170 - accuracy: 0.9942
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0169 - accuracy: 0.9942
1652/1688 [============================>.] - ETA: 0s - loss: 0.0171 - accuracy: 0.9942
1675/1688 [============================>.] - ETA: 0s - loss: 0.0171 - accuracy: 0.9942
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0172 - accuracy: 0.9942 - val_loss: 0.0537 - val_accuracy: 0.9883
Epoch 10/10
1/1688 [..............................] - ETA: 3s - loss: 8.7430e-04 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0088 - accuracy: 0.9974
47/1688 [..............................] - ETA: 3s - loss: 0.0113 - accuracy: 0.9960
70/1688 [>.............................] - ETA: 3s - loss: 0.0094 - accuracy: 0.9973
93/1688 [>.............................] - ETA: 3s - loss: 0.0082 - accuracy: 0.9976
116/1688 [=>............................] - ETA: 3s - loss: 0.0079 - accuracy: 0.9978
139/1688 [=>............................] - ETA: 3s - loss: 0.0086 - accuracy: 0.9980
161/1688 [=>............................] - ETA: 3s - loss: 0.0081 - accuracy: 0.9981
184/1688 [==>...........................] - ETA: 3s - loss: 0.0089 - accuracy: 0.9976
207/1688 [==>...........................] - ETA: 3s - loss: 0.0085 - accuracy: 0.9979
230/1688 [===>..........................] - ETA: 3s - loss: 0.0090 - accuracy: 0.9976
253/1688 [===>..........................] - ETA: 3s - loss: 0.0089 - accuracy: 0.9973
275/1688 [===>..........................] - ETA: 3s - loss: 0.0087 - accuracy: 0.9975
297/1688 [====>.........................] - ETA: 3s - loss: 0.0089 - accuracy: 0.9974
320/1688 [====>.........................] - ETA: 3s - loss: 0.0091 - accuracy: 0.9974
343/1688 [=====>........................] - ETA: 3s - loss: 0.0093 - accuracy: 0.9973
366/1688 [=====>........................] - ETA: 2s - loss: 0.0093 - accuracy: 0.9972
388/1688 [=====>........................] - ETA: 2s - loss: 0.0103 - accuracy: 0.9968
410/1688 [======>.......................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9966
433/1688 [======>.......................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9966
455/1688 [=======>......................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9964
478/1688 [=======>......................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9964
500/1688 [=======>......................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9965
521/1688 [========>.....................] - ETA: 2s - loss: 0.0109 - accuracy: 0.9966
542/1688 [========>.....................] - ETA: 2s - loss: 0.0107 - accuracy: 0.9967
563/1688 [=========>....................] - ETA: 2s - loss: 0.0106 - accuracy: 0.9968
584/1688 [=========>....................] - ETA: 2s - loss: 0.0107 - accuracy: 0.9966
605/1688 [=========>....................] - ETA: 2s - loss: 0.0106 - accuracy: 0.9966
626/1688 [==========>...................] - ETA: 2s - loss: 0.0108 - accuracy: 0.9966
647/1688 [==========>...................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9964
668/1688 [==========>...................] - ETA: 2s - loss: 0.0120 - accuracy: 0.9961
689/1688 [===========>..................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9958
710/1688 [===========>..................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9956
731/1688 [===========>..................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9956
753/1688 [============>.................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9956
774/1688 [============>.................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9956
795/1688 [=============>................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9956
816/1688 [=============>................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9957
837/1688 [=============>................] - ETA: 1s - loss: 0.0131 - accuracy: 0.9957
858/1688 [==============>...............] - ETA: 1s - loss: 0.0130 - accuracy: 0.9957
879/1688 [==============>...............] - ETA: 1s - loss: 0.0131 - accuracy: 0.9956
900/1688 [==============>...............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9956
921/1688 [===============>..............] - ETA: 1s - loss: 0.0131 - accuracy: 0.9957
942/1688 [===============>..............] - ETA: 1s - loss: 0.0131 - accuracy: 0.9957
963/1688 [================>.............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9956
984/1688 [================>.............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9955
1005/1688 [================>.............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9955
1026/1688 [=================>............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9955
1047/1688 [=================>............] - ETA: 1s - loss: 0.0135 - accuracy: 0.9954
1068/1688 [=================>............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9953
1089/1688 [==================>...........] - ETA: 1s - loss: 0.0137 - accuracy: 0.9954
1110/1688 [==================>...........] - ETA: 1s - loss: 0.0137 - accuracy: 0.9953
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0138 - accuracy: 0.9953
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9951
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9951
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9952
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9952
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9951
1258/1688 [=====================>........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9951
1279/1688 [=====================>........] - ETA: 0s - loss: 0.0141 - accuracy: 0.9952
1300/1688 [======================>.......] - ETA: 0s - loss: 0.0140 - accuracy: 0.9952
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0140 - accuracy: 0.9952
1341/1688 [======================>.......] - ETA: 0s - loss: 0.0139 - accuracy: 0.9952
1361/1688 [=======================>......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9951
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9951
1400/1688 [=======================>......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9951
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9951
1440/1688 [========================>.....] - ETA: 0s - loss: 0.0144 - accuracy: 0.9950
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1479/1688 [=========================>....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1499/1688 [=========================>....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0144 - accuracy: 0.9950
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0144 - accuracy: 0.9950
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9950
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1637/1688 [============================>.] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1657/1688 [============================>.] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1677/1688 [============================>.] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0146 - accuracy: 0.9950 - val_loss: 0.0524 - val_accuracy: 0.9867
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9868000149726868
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
model_quantized.summary()
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:467: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate.\
1/1024 [..............................] - ETA: 3:17
56/1024 [>.............................] - ETA: 0s
114/1024 [==>...........................] - ETA: 0s
170/1024 [===>..........................] - ETA: 0s
226/1024 [=====>........................] - ETA: 0s
282/1024 [=======>......................] - ETA: 0s
338/1024 [========>.....................] - ETA: 0s
395/1024 [==========>...................] - ETA: 0s
452/1024 [============>.................] - ETA: 0s
509/1024 [=============>................] - ETA: 0s
565/1024 [===============>..............] - ETA: 0s
620/1024 [=================>............] - ETA: 0s
677/1024 [==================>...........] - ETA: 0s
734/1024 [====================>.........] - ETA: 0s
790/1024 [======================>.......] - ETA: 0s
846/1024 [=======================>......] - ETA: 0s
902/1024 [=========================>....] - ETA: 0s
958/1024 [===========================>..] - ETA: 0s
1013/1024 [============================>.] - ETA: 0s
1024/1024 [==============================] - 1s 897us/step
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9815999865531921
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9868999719619751
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.984499990940094
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:05:09 - loss: 0.0037 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0319 - accuracy: 0.9915
21/1688 [..............................] - ETA: 8s - loss: 0.0258 - accuracy: 0.9896
31/1688 [..............................] - ETA: 8s - loss: 0.0218 - accuracy: 0.9909
41/1688 [..............................] - ETA: 8s - loss: 0.0204 - accuracy: 0.9924
52/1688 [..............................] - ETA: 8s - loss: 0.0174 - accuracy: 0.9934
62/1688 [>.............................] - ETA: 8s - loss: 0.0154 - accuracy: 0.9945
72/1688 [>.............................] - ETA: 8s - loss: 0.0155 - accuracy: 0.9939
82/1688 [>.............................] - ETA: 8s - loss: 0.0163 - accuracy: 0.9935
92/1688 [>.............................] - ETA: 8s - loss: 0.0178 - accuracy: 0.9932
102/1688 [>.............................] - ETA: 8s - loss: 0.0169 - accuracy: 0.9936
112/1688 [>.............................] - ETA: 7s - loss: 0.0169 - accuracy: 0.9936
122/1688 [=>............................] - ETA: 7s - loss: 0.0157 - accuracy: 0.9941
132/1688 [=>............................] - ETA: 7s - loss: 0.0160 - accuracy: 0.9941
143/1688 [=>............................] - ETA: 7s - loss: 0.0163 - accuracy: 0.9939
153/1688 [=>............................] - ETA: 7s - loss: 0.0161 - accuracy: 0.9939
163/1688 [=>............................] - ETA: 7s - loss: 0.0160 - accuracy: 0.9937
174/1688 [==>...........................] - ETA: 7s - loss: 0.0156 - accuracy: 0.9939
184/1688 [==>...........................] - ETA: 7s - loss: 0.0153 - accuracy: 0.9939
195/1688 [==>...........................] - ETA: 7s - loss: 0.0147 - accuracy: 0.9942
205/1688 [==>...........................] - ETA: 7s - loss: 0.0148 - accuracy: 0.9942
216/1688 [==>...........................] - ETA: 7s - loss: 0.0152 - accuracy: 0.9942
226/1688 [===>..........................] - ETA: 7s - loss: 0.0146 - accuracy: 0.9945
237/1688 [===>..........................] - ETA: 7s - loss: 0.0142 - accuracy: 0.9946
247/1688 [===>..........................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9946
257/1688 [===>..........................] - ETA: 7s - loss: 0.0140 - accuracy: 0.9945
267/1688 [===>..........................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9947
277/1688 [===>..........................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9947
287/1688 [====>.........................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9948
297/1688 [====>.........................] - ETA: 7s - loss: 0.0130 - accuracy: 0.9949
307/1688 [====>.........................] - ETA: 6s - loss: 0.0127 - accuracy: 0.9951
318/1688 [====>.........................] - ETA: 6s - loss: 0.0129 - accuracy: 0.9951
328/1688 [====>.........................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9950
339/1688 [=====>........................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9949
349/1688 [=====>........................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9950
359/1688 [=====>........................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9950
369/1688 [=====>........................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9951
379/1688 [=====>........................] - ETA: 6s - loss: 0.0130 - accuracy: 0.9951
390/1688 [=====>........................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9951
400/1688 [======>.......................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9952
410/1688 [======>.......................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9951
420/1688 [======>.......................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9952
430/1688 [======>.......................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9952
440/1688 [======>.......................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9951
450/1688 [======>.......................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9952
460/1688 [=======>......................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9952
470/1688 [=======>......................] - ETA: 6s - loss: 0.0129 - accuracy: 0.9953
480/1688 [=======>......................] - ETA: 6s - loss: 0.0129 - accuracy: 0.9953
490/1688 [=======>......................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9953
500/1688 [=======>......................] - ETA: 5s - loss: 0.0129 - accuracy: 0.9954
510/1688 [========>.....................] - ETA: 5s - loss: 0.0128 - accuracy: 0.9954
520/1688 [========>.....................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9955
530/1688 [========>.....................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9955
540/1688 [========>.....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9955
551/1688 [========>.....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9955
561/1688 [========>.....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9955
571/1688 [=========>....................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9956
581/1688 [=========>....................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9956
591/1688 [=========>....................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9956
601/1688 [=========>....................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9957
612/1688 [=========>....................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9957
622/1688 [==========>...................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9957
632/1688 [==========>...................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9958
642/1688 [==========>...................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9959
653/1688 [==========>...................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9959
664/1688 [==========>...................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9959
674/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9959
684/1688 [===========>..................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9960
694/1688 [===========>..................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9960
704/1688 [===========>..................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9960
714/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9961
725/1688 [===========>..................] - ETA: 4s - loss: 0.0115 - accuracy: 0.9961
735/1688 [============>.................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9961
746/1688 [============>.................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9961
756/1688 [============>.................] - ETA: 4s - loss: 0.0113 - accuracy: 0.9962
766/1688 [============>.................] - ETA: 4s - loss: 0.0113 - accuracy: 0.9962
776/1688 [============>.................] - ETA: 4s - loss: 0.0113 - accuracy: 0.9962
786/1688 [============>.................] - ETA: 4s - loss: 0.0112 - accuracy: 0.9962
797/1688 [=============>................] - ETA: 4s - loss: 0.0111 - accuracy: 0.9962
807/1688 [=============>................] - ETA: 4s - loss: 0.0111 - accuracy: 0.9962
817/1688 [=============>................] - ETA: 4s - loss: 0.0110 - accuracy: 0.9963
827/1688 [=============>................] - ETA: 4s - loss: 0.0110 - accuracy: 0.9963
838/1688 [=============>................] - ETA: 4s - loss: 0.0110 - accuracy: 0.9963
849/1688 [==============>...............] - ETA: 4s - loss: 0.0111 - accuracy: 0.9962
859/1688 [==============>...............] - ETA: 4s - loss: 0.0110 - accuracy: 0.9963
869/1688 [==============>...............] - ETA: 4s - loss: 0.0110 - accuracy: 0.9963
879/1688 [==============>...............] - ETA: 4s - loss: 0.0109 - accuracy: 0.9963
889/1688 [==============>...............] - ETA: 4s - loss: 0.0109 - accuracy: 0.9963
900/1688 [==============>...............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
911/1688 [===============>..............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9964
922/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
932/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
943/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
953/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
963/1688 [================>.............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
973/1688 [================>.............] - ETA: 3s - loss: 0.0109 - accuracy: 0.9964
983/1688 [================>.............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
994/1688 [================>.............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
1004/1688 [================>.............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
1014/1688 [=================>............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9964
1024/1688 [=================>............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9964
1034/1688 [=================>............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9964
1044/1688 [=================>............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9964
1054/1688 [=================>............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9964
1064/1688 [=================>............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9964
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0106 - accuracy: 0.9965
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0106 - accuracy: 0.9965
1095/1688 [==================>...........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9965
1105/1688 [==================>...........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9966
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0104 - accuracy: 0.9966
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1135/1688 [===================>..........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1145/1688 [===================>..........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9967
1156/1688 [===================>..........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1166/1688 [===================>..........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9967
1176/1688 [===================>..........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9967
1186/1688 [====================>.........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9967
1196/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9967
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9968
1216/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9967
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9968
1236/1688 [====================>.........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9968
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9968
1256/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9968
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9968
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0099 - accuracy: 0.9968
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0098 - accuracy: 0.9969
1296/1688 [======================>.......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1306/1688 [======================>.......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1316/1688 [======================>.......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1327/1688 [======================>.......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1337/1688 [======================>.......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1347/1688 [======================>.......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1357/1688 [=======================>......] - ETA: 1s - loss: 0.0097 - accuracy: 0.9969
1368/1688 [=======================>......] - ETA: 1s - loss: 0.0097 - accuracy: 0.9969
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9968
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0099 - accuracy: 0.9969
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0097 - accuracy: 0.9969
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1469/1688 [=========================>....] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0098 - accuracy: 0.9969
1490/1688 [=========================>....] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1501/1688 [=========================>....] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0099 - accuracy: 0.9969
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9969
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0099 - accuracy: 0.9969
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9969
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9969
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0099 - accuracy: 0.9969
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0099 - accuracy: 0.9969
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0100 - accuracy: 0.9968
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1633/1688 [============================>.] - ETA: 0s - loss: 0.0100 - accuracy: 0.9968
1643/1688 [============================>.] - ETA: 0s - loss: 0.0100 - accuracy: 0.9968
1653/1688 [============================>.] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1663/1688 [============================>.] - ETA: 0s - loss: 0.0099 - accuracy: 0.9968
1673/1688 [============================>.] - ETA: 0s - loss: 0.0099 - accuracy: 0.9969
1683/1688 [============================>.] - ETA: 0s - loss: 0.0099 - accuracy: 0.9969
1688/1688 [==============================] - 15s 6ms/step - loss: 0.0098 - accuracy: 0.9969 - val_loss: 0.0453 - val_accuracy: 0.9892
Epoch 2/5
1/1688 [..............................] - ETA: 9s - loss: 0.0014 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0027 - accuracy: 1.0000
22/1688 [..............................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9986
32/1688 [..............................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9990
42/1688 [..............................] - ETA: 8s - loss: 0.0072 - accuracy: 0.9978
53/1688 [..............................] - ETA: 8s - loss: 0.0072 - accuracy: 0.9976
63/1688 [>.............................] - ETA: 8s - loss: 0.0082 - accuracy: 0.9975
73/1688 [>.............................] - ETA: 8s - loss: 0.0091 - accuracy: 0.9970
83/1688 [>.............................] - ETA: 8s - loss: 0.0085 - accuracy: 0.9974
93/1688 [>.............................] - ETA: 8s - loss: 0.0083 - accuracy: 0.9973
103/1688 [>.............................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9976
113/1688 [=>............................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9975
123/1688 [=>............................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9975
133/1688 [=>............................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9974
143/1688 [=>............................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9974
154/1688 [=>............................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9976
164/1688 [=>............................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9975
174/1688 [==>...........................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9977
184/1688 [==>...........................] - ETA: 7s - loss: 0.0070 - accuracy: 0.9976
194/1688 [==>...........................] - ETA: 7s - loss: 0.0068 - accuracy: 0.9977
204/1688 [==>...........................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9979
214/1688 [==>...........................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9980
224/1688 [==>...........................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9979
234/1688 [===>..........................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9980
244/1688 [===>..........................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9981
254/1688 [===>..........................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9982
264/1688 [===>..........................] - ETA: 7s - loss: 0.0061 - accuracy: 0.9981
274/1688 [===>..........................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9979
284/1688 [====>.........................] - ETA: 7s - loss: 0.0066 - accuracy: 0.9979
295/1688 [====>.........................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9980
305/1688 [====>.........................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9981
315/1688 [====>.........................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9981
325/1688 [====>.........................] - ETA: 6s - loss: 0.0066 - accuracy: 0.9980
335/1688 [====>.........................] - ETA: 6s - loss: 0.0067 - accuracy: 0.9979
346/1688 [=====>........................] - ETA: 6s - loss: 0.0066 - accuracy: 0.9979
357/1688 [=====>........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9980
368/1688 [=====>........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9980
379/1688 [=====>........................] - ETA: 6s - loss: 0.0066 - accuracy: 0.9979
389/1688 [=====>........................] - ETA: 6s - loss: 0.0067 - accuracy: 0.9978
399/1688 [======>.......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9977
410/1688 [======>.......................] - ETA: 6s - loss: 0.0069 - accuracy: 0.9977
420/1688 [======>.......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9978
430/1688 [======>.......................] - ETA: 6s - loss: 0.0069 - accuracy: 0.9978
440/1688 [======>.......................] - ETA: 6s - loss: 0.0069 - accuracy: 0.9978
450/1688 [======>.......................] - ETA: 6s - loss: 0.0069 - accuracy: 0.9978
461/1688 [=======>......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9979
471/1688 [=======>......................] - ETA: 6s - loss: 0.0069 - accuracy: 0.9978
481/1688 [=======>......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9979
491/1688 [=======>......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9979
501/1688 [=======>......................] - ETA: 5s - loss: 0.0069 - accuracy: 0.9979
512/1688 [========>.....................] - ETA: 5s - loss: 0.0069 - accuracy: 0.9979
523/1688 [========>.....................] - ETA: 5s - loss: 0.0070 - accuracy: 0.9978
533/1688 [========>.....................] - ETA: 5s - loss: 0.0069 - accuracy: 0.9979
543/1688 [========>.....................] - ETA: 5s - loss: 0.0068 - accuracy: 0.9979
554/1688 [========>.....................] - ETA: 5s - loss: 0.0068 - accuracy: 0.9979
564/1688 [=========>....................] - ETA: 5s - loss: 0.0069 - accuracy: 0.9979
574/1688 [=========>....................] - ETA: 5s - loss: 0.0069 - accuracy: 0.9979
584/1688 [=========>....................] - ETA: 5s - loss: 0.0069 - accuracy: 0.9979
594/1688 [=========>....................] - ETA: 5s - loss: 0.0068 - accuracy: 0.9979
604/1688 [=========>....................] - ETA: 5s - loss: 0.0068 - accuracy: 0.9980
614/1688 [=========>....................] - ETA: 5s - loss: 0.0067 - accuracy: 0.9980
624/1688 [==========>...................] - ETA: 5s - loss: 0.0066 - accuracy: 0.9980
635/1688 [==========>...................] - ETA: 5s - loss: 0.0066 - accuracy: 0.9980
645/1688 [==========>...................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9981
655/1688 [==========>...................] - ETA: 5s - loss: 0.0066 - accuracy: 0.9980
665/1688 [==========>...................] - ETA: 5s - loss: 0.0066 - accuracy: 0.9981
675/1688 [==========>...................] - ETA: 5s - loss: 0.0066 - accuracy: 0.9981
685/1688 [===========>..................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9981
695/1688 [===========>..................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9982
705/1688 [===========>..................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9981
715/1688 [===========>..................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9981
726/1688 [===========>..................] - ETA: 4s - loss: 0.0066 - accuracy: 0.9981
736/1688 [============>.................] - ETA: 4s - loss: 0.0068 - accuracy: 0.9980
746/1688 [============>.................] - ETA: 4s - loss: 0.0067 - accuracy: 0.9981
756/1688 [============>.................] - ETA: 4s - loss: 0.0067 - accuracy: 0.9981
766/1688 [============>.................] - ETA: 4s - loss: 0.0067 - accuracy: 0.9981
776/1688 [============>.................] - ETA: 4s - loss: 0.0067 - accuracy: 0.9981
786/1688 [============>.................] - ETA: 4s - loss: 0.0066 - accuracy: 0.9981
797/1688 [=============>................] - ETA: 4s - loss: 0.0066 - accuracy: 0.9982
807/1688 [=============>................] - ETA: 4s - loss: 0.0066 - accuracy: 0.9982
817/1688 [=============>................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9982
827/1688 [=============>................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9982
837/1688 [=============>................] - ETA: 4s - loss: 0.0064 - accuracy: 0.9982
848/1688 [==============>...............] - ETA: 4s - loss: 0.0064 - accuracy: 0.9983
858/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
868/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
878/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
889/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
899/1688 [==============>...............] - ETA: 3s - loss: 0.0063 - accuracy: 0.9983
909/1688 [===============>..............] - ETA: 3s - loss: 0.0063 - accuracy: 0.9983
919/1688 [===============>..............] - ETA: 3s - loss: 0.0063 - accuracy: 0.9983
930/1688 [===============>..............] - ETA: 3s - loss: 0.0064 - accuracy: 0.9983
940/1688 [===============>..............] - ETA: 3s - loss: 0.0066 - accuracy: 0.9982
950/1688 [===============>..............] - ETA: 3s - loss: 0.0065 - accuracy: 0.9983
960/1688 [================>.............] - ETA: 3s - loss: 0.0067 - accuracy: 0.9982
970/1688 [================>.............] - ETA: 3s - loss: 0.0067 - accuracy: 0.9982
980/1688 [================>.............] - ETA: 3s - loss: 0.0067 - accuracy: 0.9982
990/1688 [================>.............] - ETA: 3s - loss: 0.0066 - accuracy: 0.9982
1001/1688 [================>.............] - ETA: 3s - loss: 0.0066 - accuracy: 0.9983
1011/1688 [================>.............] - ETA: 3s - loss: 0.0066 - accuracy: 0.9983
1022/1688 [=================>............] - ETA: 3s - loss: 0.0066 - accuracy: 0.9983
1032/1688 [=================>............] - ETA: 3s - loss: 0.0065 - accuracy: 0.9983
1042/1688 [=================>............] - ETA: 3s - loss: 0.0065 - accuracy: 0.9983
1052/1688 [=================>............] - ETA: 3s - loss: 0.0065 - accuracy: 0.9983
1062/1688 [=================>............] - ETA: 3s - loss: 0.0065 - accuracy: 0.9983
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0065 - accuracy: 0.9983
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0064 - accuracy: 0.9983
1094/1688 [==================>...........] - ETA: 2s - loss: 0.0064 - accuracy: 0.9983
1105/1688 [==================>...........] - ETA: 2s - loss: 0.0064 - accuracy: 0.9983
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0064 - accuracy: 0.9983
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9983
1135/1688 [===================>..........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9983
1146/1688 [===================>..........] - ETA: 2s - loss: 0.0064 - accuracy: 0.9984
1156/1688 [===================>..........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1166/1688 [===================>..........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1176/1688 [===================>..........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1187/1688 [====================>.........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9983
1197/1688 [====================>.........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1207/1688 [====================>.........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9983
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9983
1258/1688 [=====================>........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1289/1688 [=====================>........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9984
1299/1688 [======================>.......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9984
1310/1688 [======================>.......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9984
1320/1688 [======================>.......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9984
1330/1688 [======================>.......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9984
1340/1688 [======================>.......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1350/1688 [======================>.......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1360/1688 [=======================>......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0063 - accuracy: 0.9985
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0063 - accuracy: 0.9985
1422/1688 [========================>.....] - ETA: 1s - loss: 0.0063 - accuracy: 0.9985
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0063 - accuracy: 0.9985
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0064 - accuracy: 0.9985
1463/1688 [=========================>....] - ETA: 1s - loss: 0.0064 - accuracy: 0.9984
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0064 - accuracy: 0.9984
1483/1688 [=========================>....] - ETA: 1s - loss: 0.0065 - accuracy: 0.9984
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0065 - accuracy: 0.9984
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0064 - accuracy: 0.9984
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0064 - accuracy: 0.9984
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0064 - accuracy: 0.9984
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0064 - accuracy: 0.9985
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0063 - accuracy: 0.9985
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0064 - accuracy: 0.9985
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0064 - accuracy: 0.9985
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0063 - accuracy: 0.9985
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0064 - accuracy: 0.9985
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0064 - accuracy: 0.9985
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0064 - accuracy: 0.9984
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0064 - accuracy: 0.9984
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0064 - accuracy: 0.9984
1639/1688 [============================>.] - ETA: 0s - loss: 0.0063 - accuracy: 0.9984
1650/1688 [============================>.] - ETA: 0s - loss: 0.0063 - accuracy: 0.9984
1660/1688 [============================>.] - ETA: 0s - loss: 0.0063 - accuracy: 0.9985
1670/1688 [============================>.] - ETA: 0s - loss: 0.0063 - accuracy: 0.9985
1680/1688 [============================>.] - ETA: 0s - loss: 0.0063 - accuracy: 0.9985
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0063 - accuracy: 0.9985 - val_loss: 0.0488 - val_accuracy: 0.9883
Epoch 3/5
1/1688 [..............................] - ETA: 8s - loss: 1.5788e-04 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9974
22/1688 [..............................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9986
32/1688 [..............................] - ETA: 8s - loss: 0.0038 - accuracy: 0.9990
43/1688 [..............................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9993
53/1688 [..............................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9994
64/1688 [>.............................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9985
74/1688 [>.............................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9983
84/1688 [>.............................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9985
94/1688 [>.............................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9987
104/1688 [>.............................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9988
114/1688 [=>............................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9989
125/1688 [=>............................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9990
136/1688 [=>............................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
146/1688 [=>............................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
156/1688 [=>............................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
166/1688 [=>............................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9992
177/1688 [==>...........................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
187/1688 [==>...........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
197/1688 [==>...........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9990
208/1688 [==>...........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9991
218/1688 [==>...........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9991
228/1688 [===>..........................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9992
238/1688 [===>..........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9991
248/1688 [===>..........................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
258/1688 [===>..........................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9992
268/1688 [===>..........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
278/1688 [===>..........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
289/1688 [====>.........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
299/1688 [====>.........................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9993
310/1688 [====>.........................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
320/1688 [====>.........................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
330/1688 [====>.........................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
340/1688 [=====>........................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
350/1688 [=====>........................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
360/1688 [=====>........................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
370/1688 [=====>........................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
380/1688 [=====>........................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9993
390/1688 [=====>........................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
400/1688 [======>.......................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
410/1688 [======>.......................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
420/1688 [======>.......................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9993
430/1688 [======>.......................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9993
440/1688 [======>.......................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
450/1688 [======>.......................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
460/1688 [=======>......................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
470/1688 [=======>......................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
480/1688 [=======>......................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
490/1688 [=======>......................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9992
500/1688 [=======>......................] - ETA: 5s - loss: 0.0043 - accuracy: 0.9992
510/1688 [========>.....................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
520/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9992
530/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9992
540/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
550/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
560/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9992
570/1688 [=========>....................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
580/1688 [=========>....................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
591/1688 [=========>....................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
601/1688 [=========>....................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9992
611/1688 [=========>....................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
621/1688 [==========>...................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
631/1688 [==========>...................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
641/1688 [==========>...................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
651/1688 [==========>...................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
661/1688 [==========>...................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9992
671/1688 [==========>...................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9992
681/1688 [===========>..................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
692/1688 [===========>..................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9992
702/1688 [===========>..................] - ETA: 4s - loss: 0.0047 - accuracy: 0.9991
713/1688 [===========>..................] - ETA: 4s - loss: 0.0047 - accuracy: 0.9991
724/1688 [===========>..................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
734/1688 [============>.................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
744/1688 [============>.................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9992
754/1688 [============>.................] - ETA: 4s - loss: 0.0047 - accuracy: 0.9992
764/1688 [============>.................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9992
774/1688 [============>.................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9992
784/1688 [============>.................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9992
794/1688 [=============>................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9992
804/1688 [=============>................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9992
814/1688 [=============>................] - ETA: 4s - loss: 0.0045 - accuracy: 0.9992
824/1688 [=============>................] - ETA: 4s - loss: 0.0045 - accuracy: 0.9992
834/1688 [=============>................] - ETA: 4s - loss: 0.0045 - accuracy: 0.9992
844/1688 [==============>...............] - ETA: 4s - loss: 0.0045 - accuracy: 0.9992
855/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
865/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
876/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
886/1688 [==============>...............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9991
896/1688 [==============>...............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
906/1688 [===============>..............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
916/1688 [===============>..............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
927/1688 [===============>..............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
937/1688 [===============>..............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
947/1688 [===============>..............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
957/1688 [================>.............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
967/1688 [================>.............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
977/1688 [================>.............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
987/1688 [================>.............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
997/1688 [================>.............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9992
1007/1688 [================>.............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9992
1017/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9992
1028/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9992
1038/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9992
1049/1688 [=================>............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
1059/1688 [=================>............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
1069/1688 [=================>............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
1079/1688 [==================>...........] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0047 - accuracy: 0.9991
1099/1688 [==================>...........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1109/1688 [==================>...........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1119/1688 [==================>...........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1129/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1139/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1149/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1159/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1169/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9991
1179/1688 [===================>..........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9990
1189/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9990
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9990
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1240/1688 [=====================>........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9989
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1260/1688 [=====================>........] - ETA: 2s - loss: 0.0050 - accuracy: 0.9990
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9989
1280/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9989
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9989
1301/1688 [======================>.......] - ETA: 1s - loss: 0.0051 - accuracy: 0.9989
1311/1688 [======================>.......] - ETA: 1s - loss: 0.0051 - accuracy: 0.9989
1321/1688 [======================>.......] - ETA: 1s - loss: 0.0051 - accuracy: 0.9989
1331/1688 [======================>.......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9988
1341/1688 [======================>.......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9989
1351/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1361/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1381/1688 [=======================>......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9988
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1411/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9988
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0054 - accuracy: 0.9987
1483/1688 [=========================>....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0053 - accuracy: 0.9988
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0053 - accuracy: 0.9988
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9988
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9988
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0054 - accuracy: 0.9987
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0054 - accuracy: 0.9987
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9987
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9987
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9987
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9987
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0055 - accuracy: 0.9987
1635/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9987
1646/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9987
1656/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9987
1666/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9987
1676/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9987
1686/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9987
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0055 - accuracy: 0.9987 - val_loss: 0.0473 - val_accuracy: 0.9885
Epoch 4/5
1/1688 [..............................] - ETA: 8s - loss: 0.0022 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0025 - accuracy: 1.0000
23/1688 [..............................] - ETA: 8s - loss: 0.0031 - accuracy: 1.0000
33/1688 [..............................] - ETA: 8s - loss: 0.0033 - accuracy: 1.0000
44/1688 [..............................] - ETA: 8s - loss: 0.0066 - accuracy: 0.9993
54/1688 [..............................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9994
64/1688 [>.............................] - ETA: 8s - loss: 0.0067 - accuracy: 0.9990
74/1688 [>.............................] - ETA: 8s - loss: 0.0065 - accuracy: 0.9987
85/1688 [>.............................] - ETA: 8s - loss: 0.0059 - accuracy: 0.9989
95/1688 [>.............................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9990
105/1688 [>.............................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9985
115/1688 [=>............................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9984
125/1688 [=>............................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9985
135/1688 [=>............................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9984
145/1688 [=>............................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9985
155/1688 [=>............................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9986
166/1688 [=>............................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9987
176/1688 [==>...........................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9988
186/1688 [==>...........................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9988
197/1688 [==>...........................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9989
207/1688 [==>...........................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9988
217/1688 [==>...........................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9987
228/1688 [===>..........................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9988
238/1688 [===>..........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9988
248/1688 [===>..........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9987
258/1688 [===>..........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9987
268/1688 [===>..........................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9986
279/1688 [===>..........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9987
289/1688 [====>.........................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9987
300/1688 [====>.........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9987
310/1688 [====>.........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9988
321/1688 [====>.........................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9988
331/1688 [====>.........................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9988
342/1688 [=====>........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9988
352/1688 [=====>........................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9988
362/1688 [=====>........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9988
372/1688 [=====>........................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9987
383/1688 [=====>........................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9986
394/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9987
405/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9986
415/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9986
425/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9987
436/1688 [======>.......................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9986
446/1688 [======>.......................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9987
456/1688 [=======>......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9987
466/1688 [=======>......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9987
476/1688 [=======>......................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9986
487/1688 [=======>......................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9986
498/1688 [=======>......................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9986
508/1688 [========>.....................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9985
518/1688 [========>.....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9986
528/1688 [========>.....................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9985
538/1688 [========>.....................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9985
548/1688 [========>.....................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9986
558/1688 [========>.....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9986
569/1688 [=========>....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9986
579/1688 [=========>....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9987
590/1688 [=========>....................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9987
600/1688 [=========>....................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9987
610/1688 [=========>....................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9987
620/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9987
631/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
642/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
652/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
662/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
672/1688 [==========>...................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9987
682/1688 [===========>..................] - ETA: 5s - loss: 0.0052 - accuracy: 0.9987
693/1688 [===========>..................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9987
703/1688 [===========>..................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9987
714/1688 [===========>..................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9987
724/1688 [===========>..................] - ETA: 4s - loss: 0.0053 - accuracy: 0.9987
734/1688 [============>.................] - ETA: 4s - loss: 0.0053 - accuracy: 0.9987
744/1688 [============>.................] - ETA: 4s - loss: 0.0053 - accuracy: 0.9987
754/1688 [============>.................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9987
764/1688 [============>.................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9987
774/1688 [============>.................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9987
785/1688 [============>.................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9988
795/1688 [=============>................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9988
806/1688 [=============>................] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
816/1688 [=============>................] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
826/1688 [=============>................] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
836/1688 [=============>................] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
846/1688 [==============>...............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
856/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
867/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
877/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
888/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
898/1688 [==============>...............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9989
908/1688 [===============>..............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9989
918/1688 [===============>..............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
929/1688 [===============>..............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
939/1688 [===============>..............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
950/1688 [===============>..............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
960/1688 [================>.............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
971/1688 [================>.............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
981/1688 [================>.............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9989
991/1688 [================>.............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
1001/1688 [================>.............] - ETA: 3s - loss: 0.0049 - accuracy: 0.9989
1011/1688 [================>.............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9989
1022/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9989
1032/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9989
1043/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1054/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1064/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9989
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1085/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9990
1095/1688 [==================>...........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9990
1105/1688 [==================>...........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9989
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9989
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9989
1135/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1145/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1155/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1166/1688 [===================>..........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9989
1176/1688 [===================>..........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1187/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1197/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1207/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1238/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1249/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1291/1688 [=====================>........] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1301/1688 [======================>.......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1312/1688 [======================>.......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1322/1688 [======================>.......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1332/1688 [======================>.......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1342/1688 [======================>.......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1352/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1362/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1372/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0047 - accuracy: 0.9990
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0047 - accuracy: 0.9990
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1414/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0047 - accuracy: 0.9990
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0047 - accuracy: 0.9990
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1467/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1477/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1487/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1640/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1651/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1662/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1673/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1683/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0046 - accuracy: 0.9989 - val_loss: 0.0475 - val_accuracy: 0.9893
Epoch 5/5
1/1688 [..............................] - ETA: 8s - loss: 8.4676e-05 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0033 - accuracy: 1.0000
22/1688 [..............................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9986
32/1688 [..............................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9990
42/1688 [..............................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9985
53/1688 [..............................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9988
63/1688 [>.............................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9990
74/1688 [>.............................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9992
84/1688 [>.............................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9989
94/1688 [>.............................] - ETA: 8s - loss: 0.0036 - accuracy: 0.9990
105/1688 [>.............................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9991
115/1688 [=>............................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9992
125/1688 [=>............................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9992
135/1688 [=>............................] - ETA: 7s - loss: 0.0032 - accuracy: 0.9993
145/1688 [=>............................] - ETA: 7s - loss: 0.0032 - accuracy: 0.9994
155/1688 [=>............................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9992
165/1688 [=>............................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9991
175/1688 [==>...........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9991
185/1688 [==>...........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9992
195/1688 [==>...........................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9990
205/1688 [==>...........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9991
215/1688 [==>...........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9991
225/1688 [==>...........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9992
235/1688 [===>..........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9992
245/1688 [===>..........................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9991
255/1688 [===>..........................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9991
265/1688 [===>..........................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
276/1688 [===>..........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9991
286/1688 [====>.........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9991
296/1688 [====>.........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9992
306/1688 [====>.........................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
316/1688 [====>.........................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
326/1688 [====>.........................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9991
336/1688 [====>.........................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
346/1688 [=====>........................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
357/1688 [=====>........................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
367/1688 [=====>........................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
378/1688 [=====>........................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
389/1688 [=====>........................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
399/1688 [======>.......................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
409/1688 [======>.......................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
420/1688 [======>.......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
430/1688 [======>.......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
440/1688 [======>.......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
450/1688 [======>.......................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9992
460/1688 [=======>......................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9993
470/1688 [=======>......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9993
480/1688 [=======>......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
490/1688 [=======>......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
500/1688 [=======>......................] - ETA: 5s - loss: 0.0036 - accuracy: 0.9992
510/1688 [========>.....................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
521/1688 [========>.....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
531/1688 [========>.....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
542/1688 [========>.....................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
552/1688 [========>.....................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
562/1688 [========>.....................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
572/1688 [=========>....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
583/1688 [=========>....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
594/1688 [=========>....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
604/1688 [=========>....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
614/1688 [=========>....................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
624/1688 [==========>...................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
634/1688 [==========>...................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
645/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9991
656/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9991
666/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9991
676/1688 [===========>..................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9991
686/1688 [===========>..................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9990
696/1688 [===========>..................] - ETA: 4s - loss: 0.0042 - accuracy: 0.9990
706/1688 [===========>..................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
716/1688 [===========>..................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
726/1688 [===========>..................] - ETA: 4s - loss: 0.0042 - accuracy: 0.9990
736/1688 [============>.................] - ETA: 4s - loss: 0.0042 - accuracy: 0.9990
746/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
757/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9991
767/1688 [============>.................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
777/1688 [============>.................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
787/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
797/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
808/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
818/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
828/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
839/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
850/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
860/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
870/1688 [==============>...............] - ETA: 4s - loss: 0.0039 - accuracy: 0.9991
880/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
890/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
900/1688 [==============>...............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
911/1688 [===============>..............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
921/1688 [===============>..............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
931/1688 [===============>..............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
941/1688 [===============>..............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
951/1688 [===============>..............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
961/1688 [================>.............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
972/1688 [================>.............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
983/1688 [================>.............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
993/1688 [================>.............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1003/1688 [================>.............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1014/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1024/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1034/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1044/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1054/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1065/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1076/1688 [==================>...........] - ETA: 3s - loss: 0.0039 - accuracy: 0.9991
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1097/1688 [==================>...........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1107/1688 [==================>...........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1118/1688 [==================>...........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1129/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1139/1688 [===================>..........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9991
1150/1688 [===================>..........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9991
1161/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1171/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1181/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1191/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1201/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1211/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1232/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1242/1688 [=====================>........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9991
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9992
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9992
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9992
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9992
1293/1688 [=====================>........] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1303/1688 [======================>.......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1313/1688 [======================>.......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1323/1688 [======================>.......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9991
1333/1688 [======================>.......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1343/1688 [======================>.......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1353/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1363/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1373/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1393/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1414/1688 [========================>.....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1434/1688 [========================>.....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9992
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1485/1688 [=========================>....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1505/1688 [=========================>....] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1639/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1650/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1660/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1670/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1680/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0039 - accuracy: 0.9992 - val_loss: 0.0473 - val_accuracy: 0.9898
<keras.src.callbacks.History object at 0x7ff5ee2054d0>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9883000254631042
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test)
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.98580002784729
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-18.675253 -8.501675 -8.552155 0.40361765 -16.888338
-9.705791 -31.09959 10.400449 -9.264927 -1.7127333 ]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 7.049 seconds)